NAVAL  POSTGRADUATE  SCHOOL 
Monterey,  California 


THESIS 

DEVELOPMENT  AND  COMPARISON 
OF 

TACAMO  ICON  DESIGN  FORMATS 
by 

William  D.  Sanders 
March  1992 

Thesis  Advisor:  Judith  H.  Lind 

Approved  for  public  release;  distribution  is  unlimited 


Q2 


7  1 


077, 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


REPORT  DOCUMENTATION  PAGE 


la.  REPORT  SECURITY  CLASSIFICATION 
UNCLASSIFIED 


2a.  SECURITY  CLASSIFICATION  AUTHORITY 


2b.  DECLASSIFICATION/DOWNGRADING  SCHEDULE 


1b.  RESTRICTIVE  MARKINGS 


3.  DISTRIBUTION/AVAILABIUTY  OF  REPORT 
Approved  for  public  release;  distribution  is  unlimited 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 


5.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a.  NAME  OF  PERFORMING  ORGANIZATION 

Naval  Postgraduate  School 


6c.  ADDRESS  (City,  State,  and  ZIP  Code) 
Monterey,  CA  93943-5000 


7a.  NAME  OF  MONITORING  ORGANIZATION 


7b.  ADDRESS  (City,  State,  and  ZIP  Code) 


Sa.  NAME  OF  FUNDING/SPONSORING 
ORGANIZATION 


8c.  ADDRESS  (City,  State,  and  ZIP  Code) 


8b.  OFFICE  SYMBOL!  8-  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROJECT 

TASK 

NO. 

NO. 

WORK  UNIT 
ACCESSION  NO. 


11.  TITLE  (Including  Security  Classification) 

Development  and  Comparison  of  TACAMO  Icon  Design  Formats 


13b.  TIME  COVERED 
FROM  TO 


1 4.  DATE  OF  REPORT  (Year,  Month,  Day)  1 5.  Page  Count 

1992,  MARCH  118 


12  PERSONAL  AUTHOR(S) 
SANDERS,  William  Duward 


13  TYPE  OF  REPORT 

Master’s  thesis 


16.  SUPPLEMENTAL  NOTATION 

The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official  policy  or  position  of  the 
Department  of  Defense  or  the  U  S.  Government. 


18.  SUBJECT  TERMS  (Continue  on  reverse  If  necessary  and  Identify  by  block  number) 
TACAMO,  Icons,  Human-computer  interaction,  Symbols,  Labels,  Software 
interface  design,  System  Design,  Direct  manipulation  interface 


17. 

COSATI  CODES 

FIELD 

GROUP 

SUB-GROUP 

19.  ABSTRACT  (Continue  on  reverse  if  necessary  and  Identify  by  block  number) 

The  purpose  of  this  study  was  to  develop  and  evaluate  a  set  of  icons  for  the  next  generation  message  processing 
system  for  the  TACAMO  airborne  strategic  communications  platform.  An  icon  set  for  a  proposed  interface  was 
developed  through  the  use  of  an  icon  production  method  test,  that  is,  potential  users  designed  candidate  icons  that 
were  meaningful  to  them.  These  icons  were  then  refined  for  discriminabflity  via  input  from  a  user  survey.  To 
determine  if  well-developed  icons  with  alphanumeric  labels  yield  a  significant  performance  advantage  over  the  same 
icons  without  labels,  an  experiment  involving  trained  users  was  cond ucted  using  a  response  time  model.  Subtractive 
logic  was  used  to  measure  icon  identification  times  as  a  function  of  whether  they  were  or  were  not  labeled.  When 
speed  of  performance  and  rate  of  errors  were  compared,  labeling  of  icons  resulted  in  significantly  longer  response 
times,  yet  did  not  result  in  fewer  errors  for  the  tested  icon  set.  It  is  recommended  that  the  unlabeled  set  of  icons  be 
used  for  TACAMO’s  next  generation  message  processing  system,  and  that  the  icon  production  method  be  used 
more  widely  to  involve  users  in  interface  design. 


20  DISTRIBUTION/AVAILABILTIY  OF  ABSTRACT 
(X)  UNCLASSIFIED/UNLIMITED  □  SAME  AS  RPT.Q  DUC 


22a.  NAME  OF  RESPONSIBLE  INDIVIDUAL 
Judith  H.  Lind 


la.  REPORT  SECURITY  CLASSIFICATION 

Unclassified 


22b.  TELEPHONE  (Include  Araa  Coda)  22c.  OFFICE  SYMBOL 
(408)646-2543  OR/U 


DD  Form  1473,  JUN  86 


Pravloua  editions  are  obsolete. 
S/N  0102-LF-0 14-6603 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 

Unclassified 


I 


Approved  for  public  release;  distribution  is  unlimited. 


Development  and  Comparison 
of 

TACAMO  Icon  Design  Formats 
by 

William  D.  Sanders 
Lieutenant,  United  States  Navy 
B.S.,  United  States  Naval  Academy,  1984 

Submitted  in  partial  fulfillment 
of  the  requirements  for  the  degree  of 

MASTER  OF  SCIENCE  IN  OPERATIONS  ANALYSIS 

from  the 


Author: 


Approved  by: 


NAVAL  POSTGRADUATE  SCHOOL 
March  1992 


Frank  C.  Petho,  Second  Reader 


Department  of  Operations  Research 


¥ 


0 


P 


ii 


ABSTRACT 


The  purpose  of  this  study  was  to  develop  and  evaluate  a  set  of  icons  for  the 
next  generation  message  processing  system  for  the  TACAMO  airborne  strategic 
communications  platform.  An  icon  set  for  a  proposed  interface  was  developed 
through  the  use  of  an  icon  production  method  test,  that  is,  potential  users  designed 
candidate  icons  that  were  meaningful  to  them.  These  icons  were  then  refined  for 
discriminability  via  input  from  a  user  survey.  To  determine  if  well-developed  icons 
with  alphanumeric  labels  yield  a  significant  performance  advantage  over  the  same 
icons  without  labels,  an  experiment  involving  trained  users  was  conducted  using  a 
response  time  model.  Subtractive  logic  was  used  to  measure  icon  identification  times 
as  a  function  of  whether  they  were  or  were  not  labeled.  When  speed  of  performance 
and  rate  of  errors  were  compared,  labeling  of  icons  resulted  in  significantly  longer 
response  times,  yet  did  not  result  in  fewer  errors  for  the  tested  icon  set.  It  is 
recommended  that  the  unlabeled  set  of  icons  be  used  for  TACAMO’s  next 
generation  message  processing  system,  and  that  the  icon  production  method  be  used 
more  widely  to  involve  users  in  interface  design. 
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I.  INTRODUCTION 


A.  TACAMO  STRATEGIC  COMMUNICATIONS 

With  the  weight  of  the  United  States  strategic  nuclear  triad  shifted  firmly  to  the 
Navy’s  ballistic  missile  submarine  (SSBN)  fleet  (Schmitt,  1991,  and  Talbott,  1991), 
the  TACAMO1  communications  system  has  assumed  a  vital  role  in  the  world  of 
nuclear  deterrence.  TACAMO’s  primary  mission  is  to  provide  the  National 
Command  Authorities  (NCA)  a  survivable  and  endurable  means  to  command  the 
nation’s  strategic  fleet  of  ballistic  missile  submarines.  Despite  a  rapidly  changing 
world,  at  the  core  of  all  long-range  defense  planning  remains  the  need  to  minimize 
the  likelihood  of  the  extreme  case — the  all-out  nuclear  attack.  Deterrence  against 
any  attack  requires  the  assured  survival  not  only  of  powerful  retaliatory  forces,  but 
also  of  the  command  and  control  system  that  controls  them  (Ikle  and  Wohlstetter, 
1988,  p.  35). 

Conceived  during  the  height  of  cold  war  tensions,  the  TACAMO 
communications  system  was  originally  envisioned  as  a  temporary  fix  until  a  hardened 
shore-based  system  could  be  developed  to  communicate  with  the  ballistic  missile 
submarine  fleet.  But,  as  the  accuracy  of  Soviet  missiles  improved,  it  became 


'TACAMO,  a  term  used  interchangeably  for  a  specific  TACAMO  aircrew  or  the  entire  TACAMO 
communication  system,  is  derived  from  the  command  "take  charge  and  move  out."  In  1963,  the 
%  Director  of  Naval  Communications,  then  Rear  Admiral  Bernard  F.  Roeder,  U.S.  Navy,  coined  the 

phrase  at  the  first  meeting  of  the  then  highly-classified  project  to  determine  if  an  airborne  Very  Low 
Frequency  (VLF)  communications  system  was  feasible. 
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apparent  that  any  fixed  geographical  locations  would  be  vulnerable.  In  September 
1973,  TACAMO  assumed  the  key  role  of  providing  a  continuously  airborne  and 
survivable  method  of  relaying  retaliatory  orders  to  deployed  ballistic  missile 
submarines  in  the  event  of  a  nuclear  attack.  (Sanders,  1991,  p.  175) 

TACAMO  is  a  part  of  the  Worldwide  Airborne  Command  Post  System 
(WWABNCP)  which  provides  alternative  command  and  control.  The  WWABNCP 
ensures  that  the  NCA  will  retain  connectivity  with  U.S.  strategic  forces  in  the  event 
of  a  nuclear  attack.  TACAMO  receives  uplinked  encrypted  and  unencrypted 
message  traffic  over  the  spread  of  the  ladio  frequency  spectrum  (see  Figure  1.1). 


TACAMO  Communications 
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Figure  1,1.  TACAMO  Airborne  Communkv  ‘ions  Platform 
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Messages  are  sorted  and  prioritized  for  routing  by  the  Airborne  Communications 
Officer(ACO)  and  communications  crew  aboard  the  aircraft.  The  TACAMO 
Message  Processing  System  (TMPS)  is  used  for  automated  message  storage  and 
forwarding  to  the  aircraft’s  very  low  frequency  (VLF)  transmitter. 

The  next  TACAMO  communications  upgrade,  scheduled  for  the  mid-1990s, 
entails  the  modification  of  the  E-6A  aircraft  to  include  a  Military  Strategic  and 
Tactical  Relay  (MILSTAR)  system  with  the  associated  MILSTAR  Message 
Processing  System  (MMPS),  which  is  an  upgrade  of  the  older  TMPS.  MILSTAR  is 
a  joint  development  program  with  the  Air  Force  which  will  incorporate  anti-jam 
extremely-high  frequency  capabilities,  providing  the  Navy  with  additional  spread- 
spectrum  survivable  satellite  communications. 

Federman  (1988)  has  described  human  factors  guidelines  for  developing 
software  for  the  TMPS  upgrade  and  recommended  further  experimental  studies  in 
designing  display  formats  for  the  TMPS.  Budget  restrictions  have  limited  further 
studies;  current  plans  are  to  adapt  the  existing  command-line  processor  of  the  TMPS 
into  an  operating  system  for  the  MMPS  with  little  or  no  alteration.  The  study 
reported  here  has  integrated  Federman’s  recommendations  into  a  graphical  interface 
design  and  specifically  developed  and  evaluated  a  set  of  icons  for  the  TACAMO 
MMPS. 


* 
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B.  GRAPHICAL  INTERFACE  FOR  MILITARY  COMPUTER  APPLICATIONS 

1.  Performance  Criteria 

Computer  processing  power  and  high-resolution  graphics  for  militaiy  * 

computer  applications  have  improved  dramatically  during  the  past  decade.  Software 
designers  and  developers  are  turning  to  graphical  interfaces  and  display  formats 
including  icons,  because  pictorial  symbols  can  generally  be  recognized  more  rapidly 
and  accurately  than  words  (Green  and  Pew,  1978,  p.  103).  Haber  (1970),  for 
example,  suggests  that  it  is  easy  for  humans  to  recall  pictorial  stimuli  and  that  the 
capacity  of  memoiy  for  pictures  may  even  be  unlimited.  Icons  offer  established 
benefits  when  developed  with  creative  design  methods,  because  they  reduce 
memorization  of  commands  and  syntax  (Brown,  1986,  p.  87)  and  produce  quicker 
and  easier  recall  from  memoiy  when  compared  to  nonpictorial  symbols  (Florence 
and  Geiselman,  1986,  p.  404).  It  makes  sense,  therefore,  to  use  selectively  screened 
icons  to  facilitate  performance  within  graphical  interfaces. 

Replacing  the  vague  and  misleading  notion  of  "user  friendliness"  with 
measurable  human  factors  performance  criteria  should  be  a  top  priority  of  human- 
computer  interface  design.  Shneiderman  (1987)  proposes  that  debates  over  user 
friendliness  can  be  avoided  by  using  five  quantitative  acceptance  criteria: 

1.  Subjective  acceptance  by  user. 

2.  Initial  training  time. 

3.  Retention  of  skills  over  time. 
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4.  Speed  of  performance. 

5.  Rate  of  errors  by  the  user. 

Once  a  decision  about  the  relative  importance  of  each  of  these  human  factors 
criteria  has  been  made,  specific  acceptance  tests  should  be  established  to  guide 
designers  and  inform  prospective  users  (Shneiderman,  1987,  pp.  396-397).  While 
icons  are  only  one  of  many  interface  methods  that  can  be  employed,  they  offer  great 
potential  for  use  in  military  software  applications. 

2.  Designing  the  Optimal  Icon 

While  the  design  of  icons  cannot  be  summarized  with  a  few  rules, 
Lodding  (1983,  p.  19)  offers  guidelines  that  can  be  used  to  characterize  an  optimal 
icon: 

1.  When  first  encountering  the  icon,  a  viewer  should  be  able  to  infer  its  intended 
meaning.  Additionally,  once  learned,  the  meaning  should  be  perceived  as 
appropriate. 

2.  When  selecting  an  icon  from  a  menu  or  icon  set,  only  one  icon  should  appear 
as  appropriate. 

3.  The  icon  should  not  have  any  unnecessary  ambiguous  connotations. 

Clarity,  consistency,  familiarity,  and  simplicity  are  the  key  attributes  of  good 

icon  design  (Marcus,  1991).  Yet  rational  analysis  and  good  graphic  design  practice 
alone  cannot  guarantee  an  optimal  icon.  The  targeted  user  must  be  included  in  the 
development  process  to  establish  the  suitability  and  interpretability  of  the  design 
(Cahill,  1975,  p.  380). 
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3.  Icons:  The  Label  Debate 


While  it  is  clear  that  a  careful  and  creative  design  process  is  required  to 
develop  a  successful  set  of  icons,  the  literature  indicates  a  lack  of  consensus 
regarding  the  use  of  icons,  alphanumeric  formats,  and  labels.  While  Steiner  and 
Camacho  (1989,  p.  14)  argue  that,  for  large  amounts  of  information,  users  perform 
better  with  icons  than  with  alphanumeric  formats,  Ells  and  Dewar  (1979,  p.  167) 
found  that  for  small  amounts  of  information  users  perform  just  as  well  with 
alphanumeric  formats  as  with  icons,  and  sometimes  perform  even  better. 

Kantowitz  (1985)  notes  that  redundancy,  such  as  that  supplied  by  labels, 
tends  to  slow  down  a  system  somewhat  while  raising  its  reliability;  however, 
redundant  information  perceived  through  two  sensory  channels  simultaneously  (e.g. 
the  eyes  and  ears)  actually  speeds  up  processing  time,  compared  to  cases  where 
information  is  perceived  through  a  single  channel.  In  contrast,  Pellegrino,  Siegel, 
and  Dhawan  (1975)  warn  against  the  supplementary  use  of  labels,  using  the  dual¬ 
coding  hypothesis  that  pictures  are  encoded  both  visually  and  acoustically,  while 
labels  are  primarily  acoustically  encoded  and  thus  may  distract  from  pictures.  Yet 
another  view  is  presented  by  Guastello,  Traut,  and  Korienek  (1989,  pp.  118-119), 
who  found  that  mixed  modality  icons,  consisting  of  pictorial  icons  with  alphanumeric 
labels,  are  rated  as  more  meaningful  than  icons  that  consist  of  verbal  or  pictorial 
elements  only. 

MIL-STD-1472D,  Human  Engineering  Design  Criteria  for  Military  Systems, 
Equipment  and  Facilities  (1989)  requires  the  use  of  labels  for  icons.  Paragraph 
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5.15.4.8.3  of  that  document  states,  "Where  icons  are  used  to  represent  control  actions 
in  menus,  verbal  labels  shall  be  displayed  with  each  icon  to  help  assure  that  its 
intended  meaning  will  be  understood."  The  varied  results  reported  in  the  literature 
suggest  that  the  strict  labeling  requirements  of  MIL-STD-1472D  may  be  imposing 
unrealistic  requirements  on  military  interface  designers.  Experiments  on  the  effect 
of  icon  design  format  in  military  applications  are  urgently  needed.  With  the 
controversy  surrounding  speed  of  recognition  and  rate  of  errors  by  the  user,  studies 
should  focus  on  analyses  based  on  Shneiderman’s  last  two  acceptance  criteria. 

C.  GOALS  AND  OBJECTIVES 

The  goal  of  this  study  is  to  determine  if  well-developed  icons  with  alphanumeric 
labels  yield  a  significant  performance  advantage  over  the  same  icons  without  labels. 
This  study  has  concentrated  on  a  set  of  icons  proposed  for  a  computer  display  format 
for  TACAMO’s  anticipated  message  processing  system  upgrade.  Several  objectives 
have  been  met  to  achieve  the  primary  goal. 

1.  Screen  Layout  and  Tasks.  A  prototype  display  format  for  the  MMPS  system 
was  designed,  a  set  of  screen  design  guidelines  was  documented,  and  a  set  of 
tasks  to  be  represented  on  the  screen  by  icons  was  determined. 

2.  Initial  Icon  Design.  TACAMO  aircrew  personnel  were  used  to  sketch  symbols 
and  to  suggest  labels  for  the  TACAMO  MMPS  task  descriptions.  The  resulting 
set  of  icons  was  refined  to  enhance  discriminability. 

3.  Icon  Identification  Survey.  Recognition  and  confusability  of  the  proposed  icons 
were  determined  by  surveying  TACAMO  aircrew  personnel,  to  obtain  a  final 
set  of  icons  for  evaluation. 
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4.  Icon  Design  Evaluation.  Two  icon  design  formats,  with  and  without  labels, 
were  tested  to  measure  speed  and  accuracy  of  identification  under  the  two 
conditions. 

D.  SCOPE 

The  scope  of  this  study  is  limited  a  single  set  of  icons  developed  specifically  for 
use  with  the  TACAMO  MMPS.  Speed  and  accuracy  of  recognition  were  measured 
for  the  icons  alone  and  for  icons  that  were  labeled,  to  determine  whether  a 
significant  performance  difference  could  be  noted.  Tests  were  conducted  in  a 
controlled  environment  without  stress  or  the  requirement  to  cany  out  supplementary 
tasks.  It  is  not  the  intent  of  this  study  to  validate  the  use  of  icons  in  genera]  for 
military  computer  applications.  The  goal  is  to  present  a  method  of  developing  icons, 
to  evaluate  the  resulting  icons,  and  to  make  recommendations  that  may  be  used 
during  the  development  of  the  TACAMO  MMPS. 
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II.  INTERFACE  DEVELOPMENT  AND  INITIAL  ICON  DESIGN 


%  A.  TACAMO  HUMAN  FACTORS  ANALYSIS 

Several  human  factors  analyses  have  been  carried  out  that  are  directly 
applicable  to  the  TACAMO  systems  and  especially  to  the  TACAMO  Message 
Processing  System  (TMPS).  An  analysis  of  the  TACAMO  Communications  Central 
conducted  by  Dean  and  Schlumbrecht  (1981)  details  a  typical  TACAMO  mission 
scenario,  including  descriptions  of  the  tasks  required  for  operating  Communications 
Central.  Tasks  related  to  the  preflight,  mission  assumption,  mission  relief,  landing, 
and  post  landing  phases  are  described  in  the  report. 

Dean  and  Mitchell’s  (1981)  study  outlines  human  factors  guidelines  for  the 
TACAMO  airframe  replacement  project.  General  guidelines  centered  on  reducing 
crew  station  work  load  by  designing  Communications  Central  operator  consoles  to 
minimize  stress  and  compensate  for  types  of  errors  that  are  most  likely  to  occur 
(Dean  and  Mitchell,  1981,  p.  31).  Moreover,  Dean  and  Mitchell  note  that  sleep  loss 
and  rest  disruption  are  among  the  most  severe  stresses  on  TACAMO  crew  members, 
and  many  of  the  tasks  most  vulnerable  to  sleep-loss  effects  are  centered  on  the 
operation  of  the  TMPS.  These  tasks  include  the  following: 

1.  Monitoring  tasks  (such  as  checking  radio  circuits). 

2.  Tasks  that  are  new  or  require  learning  on  the  job  (e.g.,  training  of  new  TMPS 

t  operators). 
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3.  High-workload  tasks  that  require  time-sharing  with  other  primary  and 
secondary  tasks  (e.g.,  receiving  and  transmitting  Emergency  Action  Messages). 

4.  Tasks  that  require  continuous  attention  and  steady  performance  (e.g., 
monitoring  the  TMPS  and  the  overall  communications  centra]  system). 

Given  the  in-flight  refueling  capability  of  the  E-6A,  much  longer  TACAMO 

missions  than  those  currently  flown  are  being  envisioned.  The  anticipated  resultant 

increase  in  sleep  loss  highlights  the  importance  of  crew  rest  procedures  and  of 

careful  mission  planning  for  coordinated  work-rest  cycles.  The  established  need  for 

an  upgrade  to  the  TMPS  has  reached  a  new  level  of  urgency  with  foreseeable 

TACAMO  mission  lengths  of  up  to  72  hours  (Sanders,  1991,  p.  179). 

Federman  (1988)  makes  specific  recommendations  to  correct  existing  TMPS 

deficiencies,  based  on  a  field  survey  conducted  at  the  VQ-4  squadron,  Naval  Air 

Station,  Patuxent  River,  Maryland.  Primary  areas  of  concern  are  data  entry,  data 

display,  interaction  control,  feedback,  prompts,  error  management,  and  data 

protection.  Several  factors  to  consider  in  satisfying  a  "well  human  factored  screen" 

are  outlined  by  Federman  (1988,  p.  11): 

1.  Reduction  of  memory  load. 

2.  Consistency  in  language  and  appearance  on  the  screen. 

3.  Orderly,  clean,  and  clutter- free  screens. 

4.  Understandability  of  abbreviations,  acronyms,  and  natural  language. 

5.  Use  of  headings,  captions,  instructions,  and  options  for  improved 
meaningfulness. 

6.  A  simple  way  of  retrieving  information  that  is  stored  in  the  system. 


7.  Display  of  the  "right  amount  of  information"  on  the  screen  without  overloading 
it 

8.  Help  for  the  user  when  difficulty  arises. 

B.  MMPS  PROTOTYPE  SCREEN  LAYOUT  AND  MENU  DESIGN 

Federman’s  screen  characteristics  form  a  sound  basis  for  designing  a  human- 
computer  interface  that  is  well  accepted  by  TACAMO  users.  With  the  human 
factors  deficiencies  of  the  current  TMPS  thoroughly  documented  (Federman  1988), 
careful  consideration  and  planning  should  go  into  the  development  of  software  for 
the  next  generation  MILSTAR  Message  Processing  System  (MMPS). 

The  TACAMO  MMPS,  a  dual  Rolm  Hawk/32  computer  with  increased 
memory  storage,  is  scheduled  to  replace  the  current  TMPS  after  1995  (Sanders,  1991, 
p.  181).  Current  plans  are  to  modify  the  existing  TMPS  operating  system  software 
into  an  Ada-compiled  version  for  the  MMPS  until  funding  can  be  secured  for  new 
software.  An  initial  prototype  design  for  the  TACAMO  MMPS  interface  has  been 
developed,  incorporating  the  design  recommendations  of  Federman  and  other 
TACAMO  Airborne  Communications  Officers  (ACOs)  stationed  at  the  Naval 
Postgraduate  School. 

The  TACAMO  MMPS  prototype  is  designed  to  encompass  many  of  the 
functions  currently  accomplished  with  separate  pieces  of  equipment  in 
Communications  Central.  The  majority  of  equipment  function  settings  can  be 
controlled  at  an  MMPS  terminal  with  the  addition  of  an  enhanced  data  bus.  Design 
guidelines  have  centered  on  creating  an  interface  that  includes  not  only  current 
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TMPS  functions,  but  functions  that  are  currently  accomplished  at  other  crew 
positions  in  Communications  Central.  Communications  Central  ideally  will  have  two 
MMPS  terminals,  a  primary  terminal  and  a  secondary  terminal  serving  as  a  training 
center,  document  database,  and  operational  backup.  This  second  terminal  would 
permit  conversion  of  aircraft  maintenance  manuals  and  operational  documents  into 
CD-ROM  format,  saving  significant  weight  and  storage  space  that  could  be  better 
used  for  survival  supplies  and  aircraft  spare  parts. 

Based  on  the  human  factors  analyses  discussed  earlier,  recommendations  have 
been  documented  for  the  TACAMO  MMPS  prototype  screen  layout  and  menu 
design.  These  recommendations  are  included  in  Appendix  B.  In  anticipation  of 
increased  color  and  graphics  capabilities,  a  graphical  human-computer  interface 
was  developed  for  the  MMPS  prototype.  Appendix  B.l  outlines  guidelines  for  the 
MMPS  interface,  including  design  principles,  general  display  characteristics, 
interaction  characteristics,  information  presentation  formats,  and  user  assistance. 
Menu  structure  recommendations  are  outlined  in  Appendix  B.2. 

C.  PICTORIAL  SYMBOLS  AND  INITIAL  MMPS  ICON  DESIGNS 

Over  300  years  ago,  the  German  philosopher  and  mathematician  Baron 
Gottfried  Wilhelm  Von  Leibniz  (1646-1716)  planted  the  seeds  of  symbolic  logic  and 
computer  design  (Kreiling,  1968,  p.  100).  Leibniz’  dream  was  that  someday  a 
universal  system  of  pictorial  symbols  would  exist  that  could  be  read  in  all  languages 
without  having  to  be  translated.  A  symbol  can  give  an  identity  to  a  subject  and,  by 
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repeated  use,  can  come  to  equal  it  (Holmes,  1990,  p.  11).  By  presenting  user 
commands  and  system  information  in  the  form  of  pictorial  symbols  (often  referred 
to  as  icons),  graphic  displays  reduce  the  time  and  effort  of  learning,  facilitate  user 
performance,  and  reduce  errors  (Lodding,  1983,  p.  11). 

Most  symbols  have  meaning  only  within  a  given  context  The  context  in  which 
the  icons  developed  here  are  meaningful  is  that  of  the  MMPS  terminal  and  the  tasks 
performed  there.  Appendix  B.3  lists  20  tasks  that  will  be  carried  out  on  an  MMPS; 
these  were  selected  for  initial  icon  representation.  The  task  descriptions  were 
developed  from  thumbnail  sketches  provided  by  TACAMO  ACOs  at  the  Naval 
Postgraduate  School  (NPS). 

Three  approaches  to  design  of  pictorial  symbols  were  considered  in  developing 
the  MMPS  task  icons.  The  first  approach  considered  assembling  general  ideas  of 
what  a  group  of  ACOs  think  task  icons  should  look  like.  The  second  approach 
considered  consulting  the  Symbol  Sourcebook  (Dreyfuss,  1972)  and  Handbook  of 
Pictorial  Symbols  (Modley,  1976)  to  identify  icons  for  these  tasks  based  on  symbols 
developed  by  industrial  designers  for  various  purposes.  Neither  of  these  methods, 
however,  qualifies  as  a  scientific  approach  to  the  development  of  a  satisfactory  set 
of  icons  specific  for  the  MMPS.  The  third  approach  considered  is  referred  to  as  the 
population-stereotype  production  method  (Mudd  and  Karsh,  1961;  Howell  and 
Fuchs,  1968;  Green,  1979).  This  was  the  primary  approach  selected  for  MMPS  icon 
development,  with  some  modifications,  to  generate  a  set  of  icons  that  could  be  used 
for  further  study. 


III.  ICON  PRODUCTION  METHOD  TEST 


A.  BACKGROUND 

The  production  method  technique  solicits  users  of  an  unbiased  subject 
population  to  draw  symbols  for  specific  concepts.  Karsh  and  Mudd  (1962)  compared 
the  accuracy  of  identification  of  vehicle  control  symbols  developed  by  designers  with 
and  without  data  from  the  production  method  technique.  Their  results  showed  that 
the  production-aided  symbols  were  significantly  superior  to  symbols  developed 
without  the  production  method,  indicating  that  the  "design  and  effectiveness  and 
ultimate  effectiveness  of  any  symbol  is  entirely  dependent  upon  the  prevailing 
concepts  that  the  specific  user  population  may  have  of  its  existing  equipment" 

The  production  method  for  designing  icons  is  an  effective,  low-cost  technique 
for  generating  new  symbols  representing  software  tasks  in  a  graphical  format.  There 
are  four  essential  steps  in  the  icon  production  method: 

1.  Determine  concepts  for  software  tasks  from  the  user  population. 

2.  Identify  the  subject  population  that  will  draw  icons. 

3.  Elicit  icons  from  the  subjects. 

4.  Summarize  and  analyze  the  data. 

Candidate  icons  are  then  developed  from  categories  suggested  by  the  subjects’ 
drawings.  Green  (1979,  p.77)  notes  that  subjects’  drawings  are  strongly  influenced 
by  their  limitations  as  artists  and  their  responses  should  be  viewed  as  suggestive  and 


14 


not  definitive.  A  thorough  qualitative  analysis  of  the  test  data  combined  with  human 
factors  design  strategies  will  yield  a  superior  set  of  candidate  icons. 

B.  METHOD 

1.  Questionnaire  Development 

The  TACAMO  Icon  Production  Questionnaire  presented  in  Appendix  C.  1 
was  developed  to  gather  the  data,  using  guidelines  outlined  by  Tull  and  Albaum 
(1973).  The  questionnaire  had  three  parts.  The  first  part  requested  biographical 
data.  The  second  section  solicited  respondents’  opinions  on  menu  structure 
development,  not  directly  related  to  icon  production.  The  final  section  consisted  of 
one-inch  square  boxes  in  which  the  participants  were  asked  to  sketch  proposed 
drawings  and  write  out  suggested  labels. 

2.  Participants 

A  sample  of  12  TACAMO  aircrew  personnel  was  drawn  from  the  VQ-4 
squadron,  Naval  Air  Station,  Patuxent  River,  Maryland.  Ten  participants  completed 
the  test,  while  two  aircrewmen  were  unable  to  complete  the  testing  due  to  an 
unscheduled  TACAMO  alert  launch.  All  participants  were  Airborne 
Communications  Officers  (n=6)  or  enlisted  Airborne  Communications  Supervisors 
(n=6)  and  ranged  in  age  from  22  to  36.  Various  levels  of  TACAMO  experience 
were  reflected  in  the  sample,  with  a  mean  of  2.5  years  of  experience  and  standard 
deviation  of  1.3  years.  While  the  sample  of  participants  was  not  completely  random 
due  to  the  operational  constraints  and  scheduling  of  aircrew  personnel,  the 
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participants  were  deemed  to  be  a  reasonable  cross-section  of  TACAMO  aircrew 
personnel. 

3.  Procedure 

Participants  were  assembled  as  a  group  for  testing.  An  introduction  to 
the  purpose  and  goals  of  the  study  was  presented  verbally  by  the  proctor  (Appendix 
C.2).  TACAMO  Icon  Production  Questionnaires  were  then  distributed  and 
participants  supplied  biographical  data  and  opinions  on  menu  structure  development 
MMPS  task  descriptions,  randomly-ordered,  were  then  presented  verbally  and  on  an 
overhead  projector  one  at  a  time.  Participants  were  asked  to  provide  a  drawing  and 
alphanumeric  label  in  separate  boxes  for  each  task.  Two  minutes  were  allotted  for 
each  icon  task.  Total  time  required  for  the  test  was  50  minutes. 

C.  RESULTS 

After  the  test  was  administered,  the  respondents’  drawings  and  labels  were  cut 
into  squares  and  assembled  by  task  description.  Reduced  versions  of  the 
participants’  drawings  appear  in  Appendix  C.3.  A  list  of  the  proposed  alphanumeric 
labels  is  contained  in  Appendix  C.4.  A  listing  of  all  of  the  picture  elements  that 
participants  used  to  represent  task  descriptions  is  included  as  Appendix  C.5.  Where 
an  objective  appraisal  of  the  free-response  drawings  might  be  seen  as  a  difficult  task, 
respondents  were  encouraged  to  explain  their  drawings  with  short  notes  if  they  felt 
that  their  drawings  might  be  misinterpreted. 
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The  drawings  were  created  without  much  difficulty  for  most  of  the  task 
descriptions.  A  majority  of  the  respondents  reported  that  the  test  was  challenging 
and  interesting.  Review  of  the  drawings,  labels,  and  picture  elements  indicates  that 
the  participants  were  strongly  influenced  by  the  wording  of  the  task  description. 
Despite  a  limited  level  of  artistic  experience,  respondents  developed  a  variety  of 
original  ideas  for  drawings,  presumably  based  on  their  varying  levels  of  TACAMO 
experience.  With  the  success  of  icon  development  largely  contingent  on  the 
development  of  initial  ideas,  the  production  method  was  an  important  step  in 
generating  candidate  icons  for  further  analysis. 


f 
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IV.  ICON  CANDIDATES 


A.  OVERVIEW  OF  ICON  DEVELOPMENT 

Two  or  three  versions  of  each  of  the  20  candidate  icons  (see  Appendix  D.l) 
were  developed  using  the  sketches  obtained  from  TACAMO  aircrew  personnel 
during  the  icon  production  method  test  described  in  Chapter  III.  The  original 
sketches  were  revised  to  improve  the  general  quality  of  the  icons,  while  retaining  the 
intended  users’  suggestions.  Based  on  a  compendium  of  guidelines  for  the  design  of 
icon-based  interfaces  (Gittins,  1986),  the  various  versions  of  each  icon  were  then 
studied  and  refined  until  one  final  candidate  icon  was  prepared  for  each  concept, 
based  on  icon  production  method  data,  improved  legibility,  and  a  discriminability 
analysis.  The  refined  set  of  20  icon  candidates  then  was  presented  to  an  additional 
group  of  TACAMO  aircrew  personnel  for  evaluation,  using  standard  survey 
procedures.  The  methodology  used  for  development  of  the  candidate  set  of  icons 
is  described  in  greater  detail  below. 

B.  REFINEMENT 

Legibility,  the  perceptual  quality  of  an  icon’s  structural  features,  is  central  to 
an  icon’s  capability  to  convey  meaning  (Webb,  Sorenson,  and  Lyons,  1989).  The  goal 
of  the  icon  identification  survey  was  to  test  whether  TACAMO  aircrew  personnel  ' 

could  interpret  the  meanings  of  the  candidate  MMPS  icons.  Refinements  were  made  < 
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to  the  original  TACAMO  icon  sketches  to  make  identification  and  interpretation 
easier. 

While  the  literature  on  icon  refinement  is  limited,  the  research  on  symbol 
refinement  techniques  and  effect  on  identification  is  more  extensive.  For  example, 
Green  (1979)  recommends  enhancing  pictographic  symbol  discriminability  by 
exploring  length-to-width  ratio,  frontal  plane  orientation,  symbol  element  repetition, 
strokewidth,  and  the  extent  to  which  enclosed  areas  are  filled  in.  The  results  of 
Remington  and  Williams  (1986)  suggest  that  a  symbol  set  is  in  some  respects  like  a 
list  that  must  be  learned,  indicating  that  intra-set  similarity  plays  an  important  role 
in  improving  performance  or  decreasing  response  times.  However,  training  and 
retention  of  icons  is  not  anticipated  as  a  problem  since  Hawkins,  Reising,  Woodson, 
and  Bertling  (1984,  p.  121)  have  concluded  that  even  if  symbols  are  not  intuitive  at 
first  glance,  they  can  become  easily  recognizable  after  a  brief  learning  period  and  are 
generally  robust  to  changes  in  complexity  and  polarity. 

This  study  chose  to  concentrate  on  the  findings  of  Green  and  Pew  (1978)  and 
Green  (1979)  who  suggest  that,  in  developing  pictographic  symbols  such  as  icons, 
designers  should  concentrate  on  enhancing  legibility  and  discriminability  to  raise 
recognition  and  lower  confusion.  High  recognition  and  low  confusion  lead  to 
meaningfulness,  a  forerunner  to  memorability.  Memorability  of  the  icons  is  in  turn 
a  precursor  to  efficient  operation  of  the  system  (Guastello  and  Traut,  1989,  p.  119). 

Gittir.s  (1986)  provides  an  excellent  summary  of  available  icon  design  guidelines 
drawn  from  a  variety  of  fields,  including  human  factors  and  ergonomics  (Easterby, 


19 


1970;  Caron,  Jamieson,  and  Dewar,  1980;  Shneiderman,  1980),  graphic  arts 
(Dreyfuss,  1972;  Marcus  1982),  and  computer  science  (Carroll  and  Thomas,  1982; 
Lodding,  1982).  In  summarizing  these  sources,  he  discusses  the  strength  of 
metaphors,  graphic  design  alternatives,  and  icon  implementation  factors. 

Cautioning  against  the  overuse  of  graphical  sophistication,  Gittins  (1986,  p.  538) 
notes  that  "considerable  psychological  and  human  factors  evidence  gained  from 
studies  of  pictographic  symbols  and  signs  suggests  that  simpler  icon  designs  are  as, 
or  more,  usable  than  complex  ones."  Appendix  D.2,  which  details  the  actual 
refinements  made  to  each  set  of  original  icon  sketches,  provides  an  overview  of  the 
design  techniques  used  on  the  TACAMO  icons.  The  final  set  of  candidate  icons 
used  in  the  test  is  shown  in  Figure  4.1. 

C.  DISCRIMINABILITY  ANALYSIS 

To  provide  a  performance-based  criterion  for  selecting  icons,  the  final  set  of 
candidate  icons  was  subjected  to  the  discriminability-index  formula  developed  by 
Geiselman,  Landee,  and  Christen  (1982).  The  discriminability-index  formula  is  based 
on  the  conclusion  that  symbols  are  judged  more  or  less  similar  on  the  basis  of  the 
number  of  shared  versus  unique  configural  attributes,  as  opposed  to  primitive 
attributes  (number  of  lines,  arcs,  etc.).  The  indices  obtained  from  the  formula 
provide  an  objective  predictor  in  reducing  search  times  for  a  specific  symbol  set. 
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Figure  4.1.  TACAMO  Candidate  Icons 


TRANSMIT 

Q 

o 

* 

SAVE 

Li 

□ 

% 

CUT  BLOCK 

S6 

BACK-UP 

ifT 

1 

STATUS 

m 

55 

CALENDAR 

B 

PUBLICATIONS 

TRAINING 

a 

The  number  of  common  attributes,  unique  attributes  of  each  icon,  and  unique 
attributes  of  the  sample  icon  domain  were  tallied  as  follows: 


Number  of 

where  x  is  the  number  of  the  20  icons 

c  = 

common 

=  2 

in  the  sample  icon  domain  having  the 

attributes 

configural  attribute  i. 

Number  of 

where  y  is  the  number  of  the  20  icons 

u  = 

unique 

=  2  y» 

in  the  sample  icon  domain  not  having 

attributes 

the  configural  attribute  i. 

Number  of 

where  z  is  the  number  of  common  attributes. 

s  = 

unique 

=  25  -  z, 

(There  were  25  instances  of  attributes  in 

attributes  of 

in  total  making  up  the  20  symbols  in  the 

sample  domain 

sample  domain). 
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Geiselman,  Landee,  and  Christen  (1982)  used  standardized  regression  weights  from 
their  multiple  regression  analysis  to  develop  their  discriminability-index  (D)  formula: 

Dc  =  0.07  [  Uc  +  Sc  ]  -  0.31  Cc,  (4.1) 

where  the  subscript  c  corresponds  to  a  specific  icon. 

While  Geiselman,  Landee,  and  Christen  ( 1982)  developed  their  discriminability- 
index  formula  for  specific  use  with  graphic-display  symbology,  their  research  provided 
a  logical  precursor  for  predicting  search  times  for  icons  used  in  human-computer 
interfaces.  For  example,  the  candidate  icon  for  Transmit  has  two  configural 
attributes,  an  aircraft  and  an  arrow  (refer  back  to  Figure  4.1).  The  aircraft  is  unique 
and  the  arrow  is  held  in  common  with  four  of  the  20  icons  in  the  candidate  icon  set 
shown  in  Figure  1.  Thus  from  equation  (4.1): 

DI  =  (0.07[(20-l)  +  (20-4)  +  (25-5)]}  -  {0.31[5]}  =  2.30. 
Discriminability  indices  calculated  for  the  original  20  candidate  icons  are  provided 
in  Appendix  D.3  along  the  revised  discriminability  indices  calculated  when  the  icon 
set  was  reduced  to  18  icons  (as  discussed  later). 

D.  ICON  IDENTIFICATION  SURVEY 
1.  Questionnaire  Development 

A  TACAMO  Icon  Identification  Survey,  contained  in  Appendix  D.4,  was 
developed  to  gather  recognition  and  confusion  data  on  the  candidate  icons  using 
guidelines  outlined  by  Tull  and  Albaum  (1973)  and  Green  and  Pew  (1978).  The 
survey  consisted  of  three  sections.  The  first  section  explained  the  background  and 
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purpose  of  the  survey  with  a  brief  description  of  the  respondents’  tasks.  Instructions 
were  outlined  and  biographical  data  was  requested  in  the  second  section.  The  final 
section  consisted  of  22  task  descriptions  for  matching  20  1-inch  square  candidate 
icons. 

2.  Respondents 

Surveys  were  distributed  to  60  TACAMO  aircrew  personnel  at  the  VQ-4 
squadron,  Naval  Air  Station,  Patuxent  River,  Maryland,  and  among  TACAMO 
ACOs  at  the  Naval  Postgraduate  School.  TACAMO  aircrewmen  who  participated 
in  the  icon  production  method  test  were  ineligible  to  participate  in  the  survey. 
Respondents  included  Airborne  Communications  Officers  (n=15)  or  enlisted 
Airborne  Communications  Supervisors  (n=6)  and  ranged  in  age  from  25  to  34. 
Various  levels  of  TACAMO  experience  were  reflected  in  the  sample,  with  a  mean 
of  3.5  years  of  experience  and  standard  deviation  of  0.9  years. 

3.  Task 

Survey  participants  were  asked  to  study  each  icon  and  select  the  one  best 
meaning  from  the  list  of  task  descriptions.  Letter  codes  corresponding  to  the  task 
descriptions  were  written  underneath  each  candidate  icon.  Estimated  time  to 
complete  a  survey  was  20  minutes. 

E.  RESULTS 

Results  of  the  icon  recognition  survey  were  favorable,  with  icon  recognition 
rates  high  and  confusion  between  icons  low  (see  Appendix  D.5).  Icon  recognition 
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rates  averaged  94.5%,  far  exceeding  the  minimum  75%  recognition  rate 
recommended  by  Heard  (1974).  Only  two  combinations  of  icons  fell  outside  Heard’s 
maximum  5%  confusion  criteria:  the  Retrieve  a  Message  and  Backup  a  Message  icons 
were  confused  10%  of  the  time  and  the  Status  of  Aircraft  and  Help  icons  were 
confused  14%  of  the  time. 

The  icon  recognition  survey  verified  that  candidate  icons  had  been  successfully 
refined.  The  Calendar  and  Bases  icons  were  dropped  from  the  study  after  additional 
input  from  TACAMO  ACOs  at  NPS  resulted  in  adjusted  task  priorities,  with  a 
definitive  set  of  18  TACAMO  icons  left  for  final  testing  (see  Figure  4.2). 
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Figure  4.2.  TACAMO  Icons  for  Final  Testing 
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V.  ICON  DESIGN  FORMAT  EXPERIMENT 


A.  BACKGROUND 

While  icons  that  satisfied  discriminability  criteria  could  have  been  arbitrarily 
selected  from  acceptable  candidates,  an  empirical  validation  of  symbol  effectiveness 
is  highly  desirable  (Cahill,  1975,  p.  379).  Of  specific  interest  for  this  study  are  the 
speed  of  performance  and  rate  of  errors  by  the  user  for  icons  with  and  without 
alphanumeric  labels.  The  labels  for  the  TACAMO  icon  set  were  generated  in  a 
manner  similar  to  the  way  the  initial  icon  sketches  were  provided.  During  the  icon 
production  method  test,  participants  were  asked  to  provide  alphanumeric  labels  to 
represent  task  descriptions  (see  Appendix  C.4). 

To  meet  minimum  legibility  requirements  prescribed  by  Siebert,  Kosten,  and 
Potter  (1959),  Baker  and  Nicholson  (1967),  and  Vartebedian  (1971),  a  15-point  WP 
Courier  Simplex  font  from  WordPerfect  Corporation’s  DrawPerfect ”  presentation 
graphics  software  was  used  to  label  the  icons  on  a  standard  VGA  640x480-pixel 
display  monitor.  The  resulting  label  was  10/16-inch  high,  or  approximately  10  to  12 
scan  lines  per  character  height  The  TACAMO  label  suggestions  were  revised 
slightly  in  order  to  meet  size  and  font  restrictions  and  can  be  seen  in  Figure  5.1 
(note  that  the  legibility  of  the  printed  labels  does  not  necessarily  correspond  to  the 
legibility  of  the  same  labels  on  a  graphics  display). 
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Figure  5.1.  Labeled  TACAMO  Icons  for  Final  Testing 


B.  EXPERIMENTAL  DESIGN 

The  design  used  for  this  experiment  was  a  one-way  between-subjects 
comparison  of  labeled  and  unlabeled  icon  stimuli  (see  Figures  4.2  and  5.1).  A 
between-subjects  design  was  chosen  to  avoid  confounding  with  a  training  bias  due  to 
learning  the  same  icon  set  twice.  All  subjects  received  an  18-icon  training  sequence, 
review  training,  baseline  reaction  testing,  additional  review  training,  and  icon 
recognition  testing.  Presentation  order  was  completely  randomized  along  with  icon 
position  in  the  3  x 3-icon  matrices. 
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1.  Reaction  Time  Model 


The  selection  of  skill-based  actions  generally  has  been  studied  by 
measuring  reaction  time  (Wickens,  1992,  p.  313).  The  time  to  react  in  a  situation  in 
which  any  one  of  several  signals  may  occur  must  include  time  to  complete  four 
processes  (Welford,  1980). 

1.  Reception  of  the  signal  by  a  sense  organ  and  conveyance  of  data  by  afferent 
nerves  to  the  brain. 

2.  Identification  of  the  signal. 

3.  Choice  of  the  corresponding  response. 

4.  Initiation  of  the  action  that  constitutes  the  response. 

Card,  Moran,  and  Newell  (1982)  model  the  human  as  an  "information- 
processor"  through  a  specific,  simple  model  of  perceptual,  cognitive,  and  motor 
processes.  The  Keystroke-Level  Model  developed  by  these  researchers  defines  the 
time  to  complete  a  unit  task  as  the  sum  of  acquisition  time  and  response  or 
execution  time: 

T unUlask  =  T 'acquire  +  ^execute  •  (5-l) 

For  the  icon  design  format  experiment,  acquisition  time  for  a  unit  task  depends  on 
the  time  required  to  locate  and  identify  an  icon.  Execution  time  is  the  time  required 
to  respond  manually  to  the  stimulus  and  depends  on  motor  skills  and  system 
response  times. 
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2.  Measuring  Acquisition  Time 

This  experiment  focused  on  acquisition  time.  To  measure  acquisition 
time,  it  was  necessary  to  measure  total  response  time,  then  to  subtract  out  the  time 
required  for  manual  response.  The  subtraction  logic  technique  developed  by 
Donders  (1869,  trans.  1969)  and  refined  by  Wickens  (1992,  p.  335)  was  used  for  this 
purpose.  That  is,  a  simple  baseline  reaction-time  test  (described  later)  was 
administered  to  measure  subjects’  response  times.  Average  reaction  time  for  each 
subject  was  subtracted  from  total  response  time  to  obtain  acquisition  time. 

C.  METHOD 

1.  Subjects 

Subjects  for  the  experiment  consisted  of  36  active  duty  military  officers 
enrolled  in  various  master’s  degree  programs  at  the  Naval  Postgraduate  School. 
Subjects  ranged  in  age  from  27  to  40,  with  a  mean  of  31.9  years  of  age  and  standard 
deviation  of  3.8  years.  Four  subjects  were  female  and  32  were  male;  four  subjects 
were  left-handed  and  32  were  right-handed.  Various  levels  of  computer  experience 
were  reported  by  the  subjects.  The  subjects  were  randomly  divided  into  two  groups; 
18  were  tested  using  icons  with  alphanumeric  labels  and  18  were  tested  on  the 
unlabeled  icons. 

2.  Stimuli 

The  set  of  18  TACAMO  icons  with  and  without  alphanumeric  labels  were 
used  as  test  stimuli.  In  addition  to  the  18  TACAMO  icons,  18  "distractor"  icons  with 
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and  without  alphanumeric  labels  were  also  included  to  increase  search  complexity, 
as  suggested  by  Murdock  (1982,  p.  20).  The  distractors  were  developed  from 
Modley’s  (1976)  pictorial  symbols  to  approximate  discrim  inability-index  ratings  of 
corresponding  TACAMO  icons.  Both  the  test  icons  and  the  distractor  icons  were 
presented  as  black  images  in  0.75-inch  white  squares  on  a  non-glare  light-gray 
background.  The  stimuli  were  presented  in  the  form  of  324  3x3  randomized  icon 
matrices  (see  Figure  5.2).  The  complete  sets  of  TACAMO  icons  and  distractor  icons 
used  for  testing  are  shown  in  Appendix  E.l. 

3.  Apparatus 

The  experiment  was  conducted  on  a  Unisys  386  IBM-compatible  personal 
computer  with  an  Intel  80387™  math  coprocessor,  running  at  16  mHz.  The  graphics 
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Figure  5.2.  Icon  Recognition  Test  Matrix  Including  TACAMO 
Icons  and  Distractor  Icons 
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were  displayed  on  a  14-inch  Unisys  color  VGA  640x480  monitor.  Genus  Micro¬ 
programming’s  Proteus™  prototyping  software  was  used  to  conduct  all  phases  of  the 
experiment  Reactions  were  captured  on  the  keyboard’s  numeric  keypad  keys  "1" 
through  "9."  A  time-stamping  program  written  in  C  was  used  to  capture  response 
times  from  the  Unisys  system  clock  (Appendix  E.2). 

4.  Procedure 

Complete  instructions  for  the  experiment  were  read  aloud  to  each  subject 
and  biographical  data  was  documented  (see  Appendices  E.3  and  E.4,  respectively). 
The  experiment  consisted  of  five  phases:  initial  icon  training,  review  training,  baseline 
reaction  testing,  additional  review  training,  and  icon  recognition  testing. 

a.  Training 

Icon  training  phases  presented  a  set  of  18  TACAMO  icons  with  task 
descriptions.  Training  was  self-paced  and  identical,  except  for  icon  labels,  for  all 
subjects.  Review  training  phases  allowed  for  self-paced  study  of  the  same  icons  with 
abbreviated  task  descriptions. 

b.  Baseline  Reaction  Time  Testing 

Baseline  reaction  time  testing  consisted  of  a  simple  search  for  a  black 
block  in  a  3x3  matrix  (see  Figure  5.3)  and  a  one-finger  response  corresponding  to 
the  location  of  the  stimulus  within  the  matrix.  All  subjects  used  the  index  finger  of 
the  right  hand,  starting  at  a  common  point  5.5  cm  directly  below  the  centrally-located 
"5"  key  on  the  numeric  keypad.  The  index  finger  was  then  returned  to  the  common 
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Figure  5.3.  Baseline  Reaction  Test  Matrix 


starting  point  before  continuing  to  the  next  screen  for  the  next  test.  A  total  of  27 
reaction  times  were  recorded  for  each  subject.  The  first  nine  responses  for  each 
subject  were  discarded  as  warmup  trials. 

c.  Icon  Recognition  Tests 

For  the  icon  recognition  test,  a  text-only  screen  told  the  subject  to 
observe  the  matrix  that  would  be  displayed  and  to  select  the  icon  which  best 
represented  an  abbreviated  task  description  which  was  provided  on  the  screen  (see 
Appendix  E.5).  As  recommended  by  Murdock  (1982,  pp.  20-23),  subjects  were 
instructed  to  expect  that  a  50%  mix  of  distractor  icons  would  be  randomly  included 
in  the  icon  matrices.  When  the  subject  was  ready  to  continue,  pressing  the  spacebar 
presented  a  3x3  icon  matrix.  As  with  the  baseline  reaction  time  testing  phase, 
subjects  used  their  index  fingers  to  press  the  key  of  the  numeric  keypad  which 


corresponded  to  the  described  icon,  always  starting  at  and  returning  to  the  common 
starting  point  Each  subject  was  tested  once  on  every  icon  in  the  set  Presentation 
order  and  matrix  position  were  completely  randomized  through  the  use  of  324 
screens  of  3x3  matrices. 

5.  Performance  Measures 

Three  performance  measures  were  collected  in  addition  to  the 
measurement  of  baseline  reaction  time.  Response  times  were  recorded  for  each 
numeric  keypad  key  push.  Errors  (incorrect  key  pushes)  were  recorded,  and  the 
length  of  time  the  subject  studied  the  task  descriptions  before  beginning  each  test 
was  noted. 

D.  RESULTS  AND  DISCUSSION 

1.  Subject  Variables 

An  examination  of  the  intercorrelations  for  the  subject  descriptors 
revealed  no  significant  correlations  of  baseline  reaction  time  and  icon  identification 
response  time  to  age,  sex,  or  primary  hand.  Types  of  computer  experience  also  did 
not  correlate  to  better  or  worse  performance. 

2.  Baseline  Reaction  Test 

An  analysis  of  variance  for  baseline  reaction  times  showed  the  subject 
[F35.324  =  19.23,  p  <  0.001]  and  the  icon’s  position  in  the  matrix  [FM24  =  2.65,  p  < 
0.01]  to  be  significant.  The  interaction  between  subject  and  position  was  not 
significant. 
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The  matrix-position  effect  was  compensated  for  in  the  main  icon 
recognition  test  by  complete  randomization  of  all  icons  in  the  nine  matrix  positions. 
The  subject  effect  was  minimized  by  subtracting  each  subjects’  average  baseline 
reaction  time  from  his  or  her  individual  icon  recognition  times  to  achieve  a 
handicapping  effect,  essentially  lining  up  subjects’  mean  reaction  times  while  not 
disturbing  variation  (see  Appendix  E.6).  Baseline  reaction  time  correlated  weakly 
with  total  recognition  time  (r  =  0.29).  However,  total  recognition  time  showed 
strong  correlation  (r  =  0.997)  with  icon  identification  time;  that  is,  after  baseline 
reaction  time  was  subtracted  from  the  total.  The  correlation  between  baseline 
reaction  time  and  icon  identification  time  was  very  weak  (r  =  0.08).  This  indicates 
that  subtracting  the  baseline  reaction  time  from  the  total  response  time  was  a 
satisfactory  technique  for  obtaining  icon  acquisition  and  identification  times. 

3.  Performance  Measures 

Response  times  (with  mean  baseline  reaction  times  subtracted),  error 
rates,  and  study  times  were  analyzed  with  both  one-way  analysis  of  variance  and  the 
Kruskal- Wallis  test.  The  exploratory  data  analysis  (Velleman  and  Hoaglin,  1981) 
revealed  a  pattern  of  outliers,  leading  to  the  use  of  the  Kruskal- Wallis  nonparametric 
method  (Conover,  1980).  Significant  experimental  factors  were  found  to  be  the 
subject  [H3J  =  145.03,  p  <  0.001],  icon  design  [Hl7  =  134.71,  p  <  0.001],  icon 
position  [H8  =  31.35,  p  <  0.001],  and  labeling  [H,  =  4.81,  p  <  0.03]. 

Minitab  7.2™  was  used  to  generate  95%  sign  confidence  intervals  for  all 
icons,  for  labeled  icons,  and  for  unlabeled  icons  (see  Appendix  E.7).  Figure  5.4 
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Figure  5.4.  95%  Confidence  Intervals  on  TACAMO  Icon  Identification  Times 
(Combined  Labeled  and  Unlabeled  with  Baseline  Reaction  Time  Subtracted) 

shows  the  95%  confidence  intervals  (calculated  by  the  sign  method)  for  identification 
times  for  the  set  of  18  TACAMO  icons  (combined  labeled  and  unlabeled,  after 
baseline  reaction  time  was  subtracted).  All  icons  were  recognized  with  little  response 
time  variation  with  the  exception  of  the  Files  icon,  which  fell  back  on  a  standard 
desktop  metaphor  instead  of  a  TACAMO-specific  symbol. 

Applicable  two-way  analysis  of  variance  showed  no  significant 
interactions.  Study  time  was  weakly  correlated  to  icon  identification  time  (r  =  0.21). 
The  discriminability  indices  were  also  only  weakly  correlated  (r  =  0.22)  to 
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identification  time  for  unlabeled  icons.  A  multi-factor  analysis  of  variance  model 
could  not  be  calculated  due  to  rank  deficiencies  resulting  from  five  errors. 

4.  Error  Analysis 

A  qualitative  analysis  of  errors  showed  five  incorrect  responses  (see 
Figure  5.5).  Surprisingly,  only  one  of  the  confusion  errors  occurred  with  a  subject 
using  unlabeled  icons;  the  remaining  four  errors  were  committed  during  subjects’tests 
with  labeled  icons.  Since  the  main  purpose  of  the  icon  label  was  to  provide 
redundant  information  to  avoid  confusion,  the  results  warrant  a  closer  investigation 
of  the  effect  of  alphanumeric  labels  on  icons.  The  effect  of  the  labels  on  the  tested 
icon  set  appears  not  only  to  have  a  detrimental  effect  on  speed,  but  also  to  have  no 
effect  on  reducing  errors. 

5.  Labels — A  Closer  Look 

For  the  643  data  points,  median  acquisition  time  for  icons  without  labels 
was  1319  ms;  for  icons  with  labels,  the  median  time  was  1507  ms.  Figure  5.6  shows 
the  median  identification  times  for  each  icon,  labeled  and  unlabeled,  in  the 
TACAMO  icon  set.  Comparative  results  show  that  unlabeled  icons  yielded  quicker 
response  times  for  15  of  the  18  icons.  For  four  of  these  icons,  labeling  added  over 
400  ms  to  the  median  response  time.  Of  interest  in  Figure  5.6  are  the  results  for  the 
Files  icon,  which  show  that  almost  all  of  the  sizable  variability  shown  in  Figure  5.4 
comes  from  the  labeled  version  of  the  Files  icon.  Nonparametric  sign  confidence 
intervals  for  all  icons  combined  (labeled  and  unlabeled),  for  labeled  icons,  and  for 
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Figure  5.5.  Errors  from  Icon  Identification  Test 


unlabeled  icons  are  contained  in  Appendix  E.7.  The  Kruskal- Wallis  test  statistic 
using  a  comparison  of  average  ranks  was  calculated  as  Hj  =  4.81  with  p-value  = 
0.029;  thus  the  results  are  significant  at  a  =  0.05. 

Power  was  analyzed  using  StatGraphics  5.0™  statistical  software  and 
verified  with  operating  characteristic  curves  as  described  in  Duncan  (1986). 
Assuming  normal  distributions  with  means  of  1645  ms  for  icons  without  labels  and 
1807  ms  for  icons  with  labels,  and  a  pooled  standard  deviation  of  1361  ms,  an 
analysis  of  power  shows  /?  =  0.14.  Estimating  power  with  the  nonparametric 
medians  of  1320  ms  for  icons  without  labels  and  1510  ms  for  icons  with  labels  lowers 
f 3  to  0.03.  The  null  hypothesis  that  labels  will  result  in  faster  response  times  must 
be  rejected  at  a  =  0.05.  These  results  indicate  strongly  that  the  alphanumeric 
labeling  of  well-developed  icons  does  not  significantly  improve  identification  times 
or  reduce  errors. 
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Figure  5.6.  Median  Identification  Times  for  TACAMO  Icons 


VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY  CONCLUSIONS 

This  study  provided  important  results  in  two  areas.  First,  more  rapid  response 
times  and  lower  error  rates  expected  for  icons  with  alphanumeric  labels  did  not 
materialize.  In  fact,  alphanumeric  labels  were  shown  to  increase  response  times 
significantly,  yet  did  not  result  in  a  lower  error  rate,  for  the  TACAMO  icon  set. 
Second,  actively  involving  the  targeted  user  community  in  the  development  of  icons 
was  shown  to  be  a  valid  method  of  generating  useful  icons.  The  low-cost  approach 
used  for  the  evaluation  and  refinement  of  icon  candidates  should  be  of  particular 
value  to  designers  of  future  military  human-computer  interfaces. 

B.  TACAMO  ICONS  FOR  THE  MILSTAR  MESSAGE  PROCESSING  SYSTEM 

1.  Icon  Designs 

The  primary  result  of  this  study  is  a  set  of  icons  validated  for  use  in 
TACAMO’s  MILSTAR  Message  Processing  System  (MMPS).  Use  of  these  icons  is 
expected  to  reduce  the  need  for  menu  commands  substantially.  The  icons  should 
provide  a  faster,  more  satisfactory  human-computer  interface  since  they  incorporate 
much  more  descriptive  information  using  the  same  (or  less)  physical  display  space 
(Gittins,  1986,  p.  519).  Since  the  icons  were  developed  with  the  input  of  TACAMO 
aircrew  personnel,  they  are  expected  to  be  well-suited  for  their  distinct  mission,  and 
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their  meanings  should  be  intuitively  obvious  to  the  intended  users.  It  is  strongly 
recommended  that  the  Naval  Air  Warfare  Center,  Aircraft  Division,  develop  and  test 
a  prototype  graphical  interface  for  TACAMO’s  MMPS  that  will  include  these  icons. 

2.  Labeling  of  Icons 

Labeling  the  TACAMO  icons  is  not  recommended.  This  study  has  shown 

that  labels  do  not  effectively  aid  in  identification.  Furthermore,  it  is  recommended 

that  the  Department  of  Defense  revise  MIL-STD-1472D,  Human  Engineering  Design 

Criteria  for  Military  Systems,  Equipment  and  Facilities ,  to  reflect  the  findings  of  this 

study.  Namely,  Section  5.15.4.8.3  Supplementary  verbal  labels  should  be  deleted  and 

Section  5.15.4.8.2,  Iconic  menus,  be  rewritten  as  follows: 

Iconic  interfaces.  Graphical  system  interfaces  should  incorporate  icons  to 
represent  control  options  when  practical.  Where  icons  are  used  to  represent 
control  actions  in  menus,  they  shall  be  designed  through  production  method 
testing  and  preference  surveys  of  the  targeted  user.  Icons  should  be 
supplemented  with  verbal  labels  only  when  targeted  users  are  shown  to  be 
unsuccessful  in  developing  appropriate  icons  that  map  to  control  actions. 

The  indiscriminate  use  of  supplementary  labels  with  icons  should  be  discouraged  as 

a  "band-aid"  approach  to  human  factors  engineering,  an  inadequate  substitute  for 

proper  designs. 


C.  ICON  DEVELOPMENT  METHOD 

The  icon  production  method  test  used  during  this  study  provided  a  quick  and 
direct  source  of  reasonable  designs  for  candidate  icons.  Response  to  and 
participation  in  the  drawing  of  candidate  icons  was  positive;  the  only  disappointment 
came  from  the  relatively  small  sample  size  of  TACAMO  aircrew  personnel  available 
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to  assist  in  the  process.  With  significant  benefits  achieved  at  little  cost,  it  is 
recommended  that  the  icon  production  method  technique  demonstrated  here, 
followed  by  icon  refinement  through  user  surveys,  be  used  on  a  routine  basis  by 
designers  and  developers  responsible  for  graphical  user  interfaces  for  military 
systems. 


D.  LIMITATIONS  OF  GRAPHICAL  INTERFACES 

Icons  are  not  a  panacea;  they  cannot  completely  replace  words  in  complex 
situations.  Every  set  of  new  icons  should  be  extensively  researched,  developed,  and 
tested  before  being  introduced  into  a  new  application  (Marcus,  1991).  Based  on 
experience  gained  during  this  study,  several  specific  cautions  about  developing  icons 
for  military  systems,  such  as  TACAMO’s  MMPS,  are  offered. 

1.  Before  any  interface  format  is  implemented,  a  human  factors  analysis  of  the 
system  must  be  conducted.  A  thorough  task  analysis  logically  precedes  valid 
icon  development. 

2.  An  icon  designer  should  not  fixate  on  a  particular  type  of  metaphor  (e.g.,  the 
"office  metaphor"  prevalent  in  many  of  today’s  icon  interfaces).  In  essence,  this 
shows  a  lack  of  originality  and  an  absence  of  the  use  of  thoughtful  design 
methods. 

3.  Blanket  labeling  of  icons  is  an  admission  of  failure  by  the  interface  designer. 
The  unrestricted  addition  of  alphanumeric  labels  is  a  confession  that  the 
targeted  user  will  not  be  able  to  map  icons  successfully  to  the  task  descriptions 
they  represent. 

There  are  two  major  criticisms  leveled  against  the  use  of  icons.  The  first  is  that 
the  lack  of  empirical  study  and  understanding  of  the  cognitive  mechanism  of  icon 
interpretation  makes  it  difficult  to  recommend  design  guidelines;  thus  any  particular 
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implementation  is  arrived  at  subjectively.  The  second  criticism  is  that,  while  icon 
designs  can  indicate  associations  between  underlying  data  or  functions,  this  is  often 
all  that  can  be  achieved.  That  is,  the  design  can  indicate  the  association,  but  not  the 
•  actual  underlying  feature,  because  of  the  limits  of  graphical  design  techniques. 

(Halasz  and  Moran,  1982)  This  study  has  indicated  that  both  problems  can  be 
minimized,  at  least  for  military  systems,  by  including  the  intended  users  in  the  design 
process. 

These  cautions  and  criticisms  warn  of  the  hidden  snare  of  icons — extending 
their  use  beyond  their  obvious  efficiency  as  signifiers.  Context  forms  the  foundation 
to  effective  icon  development  in  which  a  user  finds  using  an  interface  as  obvious  as, 
say,  ringing  a  doorbell  to  enter  a  house.  Icons  are  quickly  becoming  the  choice  of 
many  interface  designers,  since  they  provide  the  only  interface  in  which  the  user  does 
not  have  to  remember  dozens  of  facts  and  processes.  Yet  the  military  must  employ 
icons  judiciously,  insisting  that  each  interface  be  tailored  with  icons  specific  to  the 
mission  at  hand. 

Future  research  on  icon  development  for  military  applications  should  focus  on 
additional  complex  issues  such  as  effects  of  color,  long-term  memory  retention,  and 
developing  an  effective  icon-discriminability  index.  For  example,  many  icons  employ 
color  while  it  is  clear  that  the  thoughtless  use  of  color  can  interfere  with  and 
overload  the  user’s  perceptual  process  (Murch,  1985).  Consideration  must  be  given 
to  human  perceptual  characteristics,  physiological  factors,  cognitive  processing,  and 
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response  capability,  for  research  leading  to  the  optimum  design  of  graphical  user 
interfaces  for  military  applications. 


While  the  scope  of  this  study  was  limited  to  the  comparison  of  icons  with  and 
without  labels  for  a  set  of  icons  developed  for  the  TACAMO  MMPS,  it  is  hoped  that 
the  approach  taken  follows  the  resourceful  path  of  Jules  Henri  Poincare:  "avoiding 
the  constructing  of  useless  combinations,  while  constructing  the  useful  combinations 
which  are  in  an  infinite  minority."  Properly  developed  and  designed,  icons  deliver 
a  sophisticated  capability,  transparent  to  the  operator  who  must  concentrate  on  the 
mission  at  hand. 
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APPENDIX  A 


ABBREVIATIONS  AND  ACRONYMS 


ACO 

HF 

LOS 

MILSTAR 

MMPS 

NCA 

NPS 

SATCOM 

SSBN 

TACAMO 

TMPS 


UH E 


VHF 

VLF 

WWABNCP 


Airborne  Communications  Officer 
High  Frequency 
Line  of  Sight 

Military  Strategic  and  Tactical  Relay  System 

MILSTAR  Message  Processing  System 

National  Command  Authorities 

Naval  Postgraduate  School,  Monterey,  California 

Satellite  Communications 

Nuclear-powered  Ballistic  Missile  Submarine 

Take  Charge  and  Move  Out  Navy  Airborne  Relay 

TACAMO  Message  Processing  System 

Ultra  High  Frequency 

Video  Graphics  Adapter 

Very  High  Frequency 

Very  Low  Frequency 

Worldwide  Airborne  Command  Post  System 
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APPENDIX  B 


TACAMO  MMPS  PROTOTYPE  INTERFACE  DESIGN 


APPENDIX  B.1 


TACAMO  MMPS  INTERFACE  GUIDELINES 


1.  Goal.  The  goal  of  these  guidelines  is  to  aid  in  developing  a  human-computer 
interface  that  allows  TACAMO  aircrew  personnel  to  perform  required  tasks 
accurately  and  efficiently.  These  guidelines  are  not  meant  to  restate  or  replace 
existing  government  and  military  guidelines,  but  are  designed  as  a  basis  for 
developing  system-specific  operational  rules  for  TACAMO’s  MILSTAR  Message 
Processing  System  (MMPS),  taking  the  considerations  of  Federman  (1988)  and  the 
opinions  of  fellow  ACOs  stationed  at  NPS  into  account. 

2.  Design  Principles.  The  following  three  principles  are  recognized  by  many  sources 
(often  in  varying  forms)  as  underlying  a  satisfactory  human-computer  interface 
(Brown,  1986;  Card,  Moran,  and  Newell,  1983;  Ehrich  and  Williges,  1986; 
Shneiderman,  1986;  Helander,  1988;  Laurel,  1990;  and  Neumann,  1991).  The 
TACAMO  MMPS  prototype  was  designed  with  these  summary  guidelines  in  mind. 

a.  Efficiency.  Information  should  be  easy  to  find.  Only  information  that 
is  essentia]  for  performing  a  task  should  be  displayed,  minimizing  operator  effort  and 
reducing  memory  load.  Grouping,  placement,  and  sequence  characteristics  should 
be  used  to  display  information  in  a  way  compatible  with  the  way  operators  use  the 
information  in  a  given  task.  The  vocabulary  of  TACAMO  aircrew  personnel  should 
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determine  data  labels,  display  titles,  menu  options,  command  language,  and  error 
messages. 

b.  Consistency.  Consistency  in  format  and  operation  of  the  interface 
enhance  operator  efficiency.  Displays  can  be  searched  faster  for  crucial  information. 
In  particular,  location  of  similar  types  of  information  should  be  consistent  from 
screen  to  screen.  General  methods  of  interaction,  or  habit  patterns,  should  be  the 
same  regardless  of  where  the  operator  is  in  the  TACAMO  MMPS.  Coding,  or 
special  meanings,  such  as  "blinking  red"  for  alarms,  should  be  applied  consistently  to 
ensure  that  meanings  are  clearly  interpreted. 

c.  Feedback.  Every  operator  action  should  invoke  a  noticeable  response 
from  the  computer.  Specifically,  when  the  operator  has  made  an  error  or  performed 
an  action  that  the  TACAMO  MMPS  does  not  understand,  the  TACAMO  MMPS 
should  provide  a  message  of  explanation  and  method  of  recovery.  When  an  operator 
performs  an  action  that  has  potential  destructive  effects,  the  TACAMO  MMPS 
should  require  confirmation  before  processing  the  request.  When  the  TACAMO 
MMPS  is  busy  performing  a  lengthy  process,  it  should  notify  the  operator  through 
a  system  message  or  display  graphic.  When  it  is  appropriate  for  an  operator  to 
select  objects  of  the  display  to  manipulate,  the  system  should  indicate  that  an  object 
has  been  selected  (by  image  reversal  highlighting  or  color  coding)  so  the  operator 
knows  on  what  object  the  selected  action  will  be  directed. 


3.  General  Screen  Characteristics.  Figure  B-l.l  shows  the  TACAMO  MMPS 
prototype  display  format  A  graphical  interface  with  pull-down  menus  and  an  iconic 
interface  should  make  the  TACAMO  MMPS  simpler  and  easier  to  use  than  the 
current  TACAMO  Message  Processing  System  (TMPS)  (Hildebrand,  1991). 
Constant  monitoring  of  all  radio  circuits  was  deemed  a  high  priority,  as  was 
establishing  a  visual  message  alarm  system,  distinguishing  the  message  editing  screen, 
and  providing  an  extensive  online  help  capability. 

a.  Pull-down  Menus.  Pull-down  menus  are  displayed  as  a  horizontal  list 
of  options  at  the  top  of  the  screen.  Proposed  MMPS  menu  structure  is  listed  in 
Appendix  B.2.  The  menus  were  ordered  by  grouping  similar  tasks  as  described  by 
Harpster  (1987)  and  McDonald,  Dayton,  and  McDonald  (1988).  Surprisingly,  it 
appears  that  the  names  given  to  commands  have  little  effect  on  menu  selection  times 
(Smelcer  and  Walker,  1990).  Operator  search  time  is  significantly  shorter  for  the 
vertical  list  format  (Backs,  Walrath,  and  Hancock,  1987).  The  menu  commands 
should  be  mixed  case  with  double  spacing  between  menu  commands  (Williams, 
1988). 

b.  Iconic  Interface.  An  appropriate  icon  may  be  easier  to  recognize  and 
comprehend  than  any  amount  of  text  (Brown,  1987,  p.  91).  By  presenting  user 
command  and  system  information  in  the  form  of  icons,  the  TACAMO  MMPS  can 
capitalize  on  the  new  capabilities  of  graphic  displays,  reducing  both  time  and  effort 
of  learning  and  facilitating  performance  while  reducing  errors  (Lodding,  1983,  p.  11). 
Ordering  of  the  icons  corresponds  to  ordering  of  the  menu  commands.  While  not 
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all  menu  commands  have  a  corresponding  icon,  all  icons  have  a  corresponding  menu 
command. 

With  the  established  superiority  of  icons  over  alphanumerics  for  many 
applications  (Steiner  and  Camacho,  1989),  an  iconic  interface  is  strongly  endorsed 
as  an  alternative  interface  for  the  TACAMO  MMPS.  While  icon  interfaces  are 
aimed  primarily  at  the  novice  (Blankenberger  and  Hahn,  1991),  studies  have  shown 
that  icons  produce  faster  search  and  selection  times  (Camacho,  Steiner,  and  Berson, 
1990)  and  in  some  cases  they  are  even  preferred  by  experienced  operators  (Guastello 
and  Traut,  1989). 

c.  Monitoring  Radio  Circuits.  The  bar  at  the  bottom  of  Figure  B-l.l  will 
be  used  to  monitor  incoming  radio  traffic.  Current  system  capabilities  do  not  allow 
the  operator  to  monitor  all  of  the  circuits  received  aboard  TACAMO  at  the  TMPS 
operator  position.  Each  circuit  identifier  would  consist  of  a  small  blue  button  with 
an  abbreviated  circuit  label  in  dark  gray.  When  a  circuit  is  actively  monitored,  the 
label  would  turn  bright  white,  indicating  normal  functions.  A  circuit  under  repair  or 
malfunctioning  would  be  indicated  by  a  light  gray  button.  Incoming  traffic  would 
turn  the  labels  yellow.  Red  labels  on  yellow  buttons  with  a  "3-D  popped  circuit 
breaker"  effect  should  be  used  for  Flash  or  higher  precedence  message  traffic.  By 
clicking  on  any  button,  the  operator  should  be  able  to  directly  access  the  desired 
circuit.  The  "3-D  popped  circuit  breaker"  effect  would  reset  itself  after  the  operator 
had  inspected  the  incoming  message  traffic  to  ensure  Flash  or  higher  message  traffic 
was  promptly  handled. 
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d.  Message  Alarm  System.  A  visual  alarm  system  for  incoming  messages 
would  eliminate  the  need  for  the  current  aural  alarm  that  is  rarely  used  due  to  its 
ineffectiveness  (Federmann,  1988,  p.  2-3).  Figure  B-l.l  shows  a  half-circle  fan  in  the 

»  upper  right-hand  corner  of  the  screen.  This  half-circle  would  serve  as  a  dual-purpose 

gauge,  normally  used  to  indicate  space  remaining  on  the  fixed-disk  drive,  but  turning 
into  a  visual  alarm  for  incoming  messages.  The  half-circle  would  be  light  gray  with 
blue  right-to-left  shading  when  indicating  the  amount  of  space  remaining  on  the  fixed 
disk.  The  blue  fill  would  turn  to  amber,  to  warn  when  the  disk  capacity  exceeds  a 
specified  percentage.  The  half-circle  would  become  a  spinning  yellow-red  fan  when 
a  Flash  or  higher  precedence  message  arrives  aboard  the  aircraft. 

e.  Editing  Screen.  An  extremely  serious  error  occurs  when  the  operator 
is  confused  about  whether  the  present  screen  displays  a  message  that  has  been 
received  or  a  message  that  is  being  edited.  A  clear  distinction  between  the  two  types 
of  screens  is  definitely  needed,  as  several  TMPS  operators  have  registered  complaints 
with  the  split  screen  used  in  current  equipment  (Federman,  1988,  p.  2-2).  The 
message  monitoring  screen  should  have  a  light  yellow  background  with  text  displayed 
in  a  blue,  sans  serif  font  similar  to  that  used  with  older  teletype  machines.  The 
editing  screen  should  be  paper  white  with  a  black  serif  font  for  high  legibility 
(Oborne  and  Holton,  1988).  For  higher  resolution  screens,  light-colored  type  on 
dark  backgrounds  produces  much  slower  reading  times  (Shneiderman,  1987,  p.  360). 
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If  additional  distinction  is  needed  between  the  two  display  formats,  simulated 
perforation  tracks  could  displayed  on  the  right  and  left  edges  of  the  message 
monitoring  screen.  Operators  should  have  the  option  of  toggling  quickly  between 
screens  and  also  the  option  of  viewing  more  than  one  screen  at  a  time,  each  in  a 
separate  window.  For  message  editing  purposes,  a  temporary  "paste-up"  buffer  space 
should  be  made  available  to  cut,  store,  and  paste  sections  of  different  messages. 

f.  Screen  Colors.  In  this  proposal,  menu  bars  are  dark  blue  with  white 
lettering.  Bright  yellow  is  used  for  the  single-letter  "speed-keys"  within  each  menu 
command.  Reverse  video  (white  background  with  blue  letters)  should  be  used  to 
highlight  pending  menu  selections.  Icons  are  drawn  with  dark  blue  details  on  a  white 
background  for  discriminability.  Similar  to  the  menus,  reverse  video  (dark  blue 
background  with  white  details)  indicates  a  pending  icon  selection.  Unusable  space 
should  be  designated  with  a  separate  color,  such  as  light  blue. 
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Figure  B-l.l.  Prototype  Interface  for  TACAMO  MMPS 
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g.  Online  Help  and  Training.  Online  help  should  be  made  available  so  the 

operator  can  access  information  about  features  of  the  software  and  can  obtain  aid 
in  problem  recovery.  Easily  accessed  information  helps  the  user  cope,  facilitates 
browsing,  and  results  in  the  rapid  learning  of  new  features  or  advanced  techniques 
(Heckel,  1991,  p.  57).  Context-sensitive  help  is  especially  valuable  in  giving  operators 
quick  answers  to  specific  problems.  Prototype  testing  and  evaluation  should  provide 
detailed  documentation  of  problem  areas  in  need  of  context-sensitive  help. 

The  TACAMO  MMPS  should  be  used  extensively  for  training 
purposes.  An  ideal  TACAMO  aircraft  would  have  more  than  one  MMPS,  with  one 
terminal  available  at  all  times  for  training  or  as  a  backup  to  the  main  terminal. 
CD-ROM-based  training  and  publications  storage  would  fully  exploit  the  capabilities 
of  a  well-planned  TACAMO  MMPS,  reducing  aircraft  weight  and  enhancing 
survivability. 

4.  Interaction  Methods. 

a.  Menu  Selection.  For  the  process  of  selecting  an  item  from  a  menu, 
cursor  keys,  a  trackball,  scrolling  bars,  and  "speed-keys"  should  be  implemented. 
Cursor  keys  operate  pull-down  menus  and  can  even  control  icon  menus,  if  desired. 
Trackballs  have  been  shown  to  be  an  effective  method  of  interaction,  providing  direct 
access  by  point  and  click.  The  trackball  is  preferred  over  the  mouse  for  applications 
involving  high  stress  and  limited  space  (Shneidennan,  1987,  p.  242). 
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The  usual  wisdom  is  that  pointing  devices,  such  as  a  trackball,  mouse, 
or  joystick,  are  faster  than  keyboard  controls;  however,  studies  have  shown  that  for 
tasks  that  mix  typing  and  pointing,  keyboard  input  is  often  preferred  and  results  in 
less  muscular  strain  (Shneiderman,  1987,  p.  247).  Scrolling  bars  should  be  provided 
to  allow  operators  to  screen  message  traffic  quickly.  The  use  of  "speed  keys,"  single 
keypresses  that  access  a  command  directly  (usually  indicated  via  a  single  highlighted 
letter  on  a  menu  command),  offers  advantages  common  to  both  cursor  keys  and  the 
trackball.  Eliminating  the  requirement  of  pressing  the  Enter  key  translates  to  a  more 
direct  method  of  entry,  often  preferred  by  experienced  users. 

Menus  should  always  be  available  for  repeated  use  if  desired.  For 
example,  if  an  operator  desires  to  switch  radio  frequencies  for  multiple  circuits  and 
for  each  radio  must  access  the  frequency  control  menu  that  is  three  levels  down  and 
three  levels  up,  the  operator  will  quickly  become  annoyed.  Operators  should  always 
have  instant  access  to  the  top  level  menu  if  desired.  The  most  important  aspect  of 
menu  selection  control  is  providing  all  reasonable  methods  of  control  so  that  the 
individual  operator  can  develop  a  style  of  operation  that  he  or  she  is  most 
comfortable  with. 

b.  Data  Entry.  For  data  entry,  direct  entries  are  significantly  faster  and 
preferred  by  system  operators  (Gould,  et  al.,  1988).  This  finding  should  not 
discourage  the  use  of  pre-filled  data  fields  for  common  entries,  such  date-time-group 
entries,  latitude-longitude  entries,  and  pre-formatted  message  forms. 
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c.  Innovative  Graphical  Displays.  Electronic  Post-It  ”  notes,  scheduling 
aids,  and  other  real-world  conveniences  should  be  made  available  to  the  operator. 
Open  communication  lines  between  the  prototype  designer  and  the  actual  TACAMO 
operators  will  ensure  that  such  conveniences  will  be  provided  in  Version  1.0  and  not 
as  a  later  add-on. 

5.  Development  of  a  Prototype.  Full  development  of  a  TMPS  MMPS  prototype 
should  directly  involve  the  TACAMO  community  at  all  stages  of  testing.  Early 
evaluation  and  thorough  iterations  will  produce  a  satisfying  end  product.  The  use 
of  PC-based  design  and  prototyping  tools  will  result  in  a  better  design  process, 
allowing  the  interface  designers  to  involve  TACAMO  aircrew  personnel  actively, 
resulting  in  a  significantly  better  user  interface  design  (Koster  and  Wilkinson,  1988, 
p.  360). 

Melkus  and  Torres  (1988)  provide  excellent  guidelines  for  using  prototyping 
with  a  minimum  of  problems.  For  a  more  thorough  treatment  on  prototyping, 
Wilson  and  Rosenberg  (1988)  describe  available  tools  and  evaluation  techniques  to 
aid  in  prototyping  from  the  "tangible  speculation"  stage  to  the  final  testing  and 
analysis. 
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APPENDIX  B.2 


MMPS  MENU  STRUCTURE 


MESSAGES^ 

Transmit  =\ 

Select  message  and  parameters 
Monitor  =\ 

EHF  or  MILSTAR 

UHF 

VLF 

HF  DATA 

Queue  of  outgoing  messages 


EDIT=) 

Create 

Retrieve 

Save 

Print 

Punch  tape 
Spellcheck 
Block 

Display  message  files=| 

Flash  messages 
General  catalog 
Standard  message  forms 


STATUS 


Comm  Central=| 

Power 
AN-USC/13 
Reel  assembly 
Aircraft  system s=^ 

Status 

PMS 

TACAMO  bases 
WWABNCP=| 

Basing 

Airborne  schedule 
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EQUIPMENT=, 

Pre-Flight  tests 
Configure  Conun  Central=^ 

Set  circuits 
Set  alarms 
Verify  FTS 

Back-up  mission  data 


TRAINING^ 

PQS  training 
Crew  training 
Mission  goals 
Squadron  goals 


LIBRARY  =| 

Abbreviations  &  acronyms 

Dictionary 

Thesaurus 

Professional^^ 

COG 

TOP 

OPORDERS 

NATOPS 

Maintenance 

HELP==| 

Help 

Index 

Version 
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APPENDIX  B3 


INITIAL  TASK  DESCRIPTIONS  FOR  MMPS  ICONS 

1.  Preflight  tests  and  checklists 

2.  Display  message  catalog  (files) 

3.  TACAMO  aircrew  basing  information 

4.  Transmit  a  message 

5.  Retrieve  an  existing  message 

6.  Print  a  paper  copy  of  a  message 

7.  General  help  index  for  MMPS  software 

8.  Aircraft  systems  readiness  status  information 

9.  Configure  Comm  Central  equipment 

10.  Spell  check  a  message 

11.  Block  and  cut  a  section  of  a  message 

12.  Create  a  message 

13.  Operational,  NATOPS,  and  maintenance  publications 

14.  PQS  syllabus  training,  programmed  texts 

15.  Thesaurus 

16.  Cut  a  teletype  tape  of  a  message 

17.  Display  the  outgoing  message  queue 

18.  Backup  mission  data  from  the  hard  disk  to  floppy  diskette 

19.  Calendar  of  scheduled  events 

20.  Save  a  message 
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APPENDIX  C 


ICON  PRODUCTION  METHOD  TEST 
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APPENDIX  C.1 


TACAMO  ICON  PRODUCTION  QUESTIONNAIRE 

Name  (optional):  _ _ 

Rank/Rate:  _  Age:  _  Sex:  M  F 

TACAMO  experience: 

Crew  position(s) _ 

Aircraft  flown  _ _ 

Time  in  TACAMO  community  _ yrs.  _ mos. 

Education: 

High  School/GED 
Some  College  Courses 
Associate’s  Degree 

College  Degree _ _ 

Postgraduate  _ _ 

Computer  experience  (circle  as  many  as  applicable): 

IBM  PCs  Macintosh  Mainframe  terminals 

Amiga  Commodore  Other _ 

Subject  number _ 
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MENU  STRUCTURE  SURVEY 


For  each  question,  please  darken  the  circle  nearest  the  level  of  importance 
you  would  place  on  each  task  for  inclusion  on  the  Milstar  Message  Processing 
System. 


1.  Preflight  tests  and  checklists. 

2.  Display  message  catalog. 

3.  TACAMO  aircrew  basing  information. 

4.  TACAMO  abbreviations  and  acronyms  dictionary. 

5.  Transmit  a  message. 

6.  Chain  of  command  organization  chart. 

7.  Retrieve  an  existing  message. 

8.  Print  a  paper  copy  of  a  message. 

9.  General  help  index  for  MMPS  software. 

10.  Aircraft  systems  readiness  status  information. 

11.  Standard  dictionary. 

12.  Configure  Comm  Central  equipment. 

13.  Spell  check  a  message. 

14.  Squadron  goals. 

15.  Mission  commander  goals. 

16.  Comm  Central  readiness  status  information. 

17.  Crew  deployment  training  goals. 

18.  Monitor  a  radio  circuit. 

19.  Block  and  cut  a  section  of  a  message. 

20.  Create  a  message. 

21.  Operational  and  maintenance  publications,  NATOPS. 

22.  PQS  syllabus  training,  programmed  texts. 

23.  Thesaurus. 

24.  Cut  a  teletype  tape  of  a  message. 

25.  WWABNCP  basing  and  deployment  status  information. 

26.  Version  number  for  MMPS  software. 

27.  Display  the  outgoing  message  queue. 

28.  Backup  mission  data  from  the  hard  disk  to  floppy  diskette. 

29.  Calendar  of  scheduled  events. 

30.  Save  a  message. 
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MENU  STRUCTURE  SURVEY 


Hot  Not  As  More  Very 

Important  Important  Important  Important  Important 
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ICON  PRODUCTION  TASK  DESCRIPTIONS 
The  following  task  descriptions  were  presented  verbally  and  on  an  overhead 
projector  slide  in  this  random  order: 

1.  Preflight  tests  and  checklists. 

2.  Display  message  catalog  (files). 

3.  Create  a  message. 

4.  Operational  and  maintenance  publications,  NATOPS. 

5.  General  help  index  for  MMPS  software. 

6.  Aircraft  systems  readiness  status  information. 

7.  Block  and  cut  a  section  of  a  message. 

8.  TACAMO  aircrew  basing  information. 

9.  Retrieve  an  existing  message. 

10.  Print  a  paper  copy  of  a  message. 

11.  Display  the  outgoing  message  queue. 

12.  Transmit  a  message. 

13.  PQS  syllabus  training,  programmed  texts. 

14.  Thesaurus. 

15.  Cut  a  teletype  tape  of  a  message. 

16.  Configure  Comm  Central  equipment. 

17.  Spell  check  a  message. 

18.  Save  a  message. 

19.  Calendar  of  scheduled  events. 

20.  Backup  mission  data  from  the  hard  disk  to  floppy  diskette. 
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TACAMO  SYMBOLOGY 
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APPENDIX  C.2 


ICON  PRODUCTION  INSTRUCTIONS 

I  am  interested  in  developing  icons  for  use  in  a  graphical  interface  for  the 
next-generation  MILSTAR  Message  Processing  System  (MMPS).  Go  ahead  and  fill 
out  the  background  information  on  the  first  page  of  your  questionnaire  before  we 
begin.  I  have  focused  on  the  use  of  icons  in  the  MMPS  because  icons  seem  to  be 
recognized  more  rapidly  and  accurately  than  command  lines  that  are  currently  used 
in  the  TACAMO  Message  Processing  System  (TMPS).  Standard  symbols  and 
pictograms  are  currently  in  use  in  automobiles,  public  transit  systems,  and  highway 
signs. 

The  key  to  developing  strong  TACAMO  icons  is  your  participation  here 
today.  The  method  I’m  using  today  is  refered  to  as  the  icon  production  method 
which  has  been  used  to  succesfully  develop  Army  vehicle  controls,  automobile  labels, 
and  more  recently  Army  battlefield  symbology. 

So,  what  makes  a  good  icon?  A 
strong  pictorial  representation,  for 
example  (draw  figure  on  board): 

This  windshield  wiper  is  an  excellent 
example  of  an  icon  that  provides  a 
strong  image  with  an  image  of  limited 
complexity.  Today  we’re  going  to  try 

Control  Symbol  for  Automobile  Wiper 

and  create  symbols  for  TACAMO  icons 
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with  similar  strengths. 

Basically,  a  good  icon  will  be: 

1.  Meaningful, 

2.  Simple,  yet  bold, 

3.  Different  looking  from  those  that  already  exist — especially  those 
that  exist  on  your  nieghbor’s  paper. 

Don’t  be  afraid  to  explain  your  artwork.  This  is  not  a  test  of  your  artistic  talent.  Try 
to  draw  each  icon  as  large  as  the  space  allows.  For  each  task  description,  please 
draw  a  picture  in  the  top  box  and  an  alphanumeric  label  in  the  bottom  box  provided. 
You  will  be  given  two  minutes  to  draw  each  icon. 

Do  you  have  any  questions? 

Thank  you  for  your  time  and  cooperation. 
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APPENDIX  CJ 

ICON  PRODUCTION  METHOD  DRAWINGS 
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Preflight  tests  and  checklists  Create  a  message 
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General  help  index  for  MMPS  software  Block  and  cut  a  section  of  a  message 
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Display  the  outgoing  message  queue 


PQS  syllabus  training,  programmed  texts  Cut  a  teletype  tape  of  a  message 
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Spell  check  a  message  Calendar  of  scheduled  events 
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APPENDIX  C.4 


LABELS  FROM  ICON  PRODUCTION  METHOD 


Task  description  Proposed  labels  (responses) 

1.  Preflight  tests  and  checklists.  PREFLT  (5) 

PREFL 

PREFLIGHT 

PFCL 

CL 

LISTS 


2.  Display  message  catalog  (files).  DMC  (4) 

CAT  (2) 

FILES(2) 
CATALOG 
MESSAGE 
MSG  CATALOG 


3.  Create  a  message. 


CREATE  (4) 
CREATE  MSG  (2) 
NEW  (2) 

NEW  MSG  (2) 
MSG 


4.  Operational  and  maintenance 
publications,  NATOPS. 


PUBS  (6) 
NATOPS  (4) 


5.  General  help  index  for  MMPS  software.  HELP  (6) 

HLP 

GEN  HELP 

MMPS 

INDEX 


6.  Aircraft  systems  readiness  status 
information. 


ACFT  STATUS  (3) 
ACFT 

A/C  STATUS 
SYSSTAT 
STATS 
STS 
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7.  Block  and  cut  a  section  of  a  message.  CUT  (7) 

BLOCK  (5) 

BLK 

PASTE 

INPUT/OUTPUT 

MSG 

8.  TACAMO  aircrew  basing  information.  BASING  (5) 

BASES  (2) 
TACBASING 
TACBAS 
BSE 

9.  Retrieve  an  existing  message.  RETRIEVE  (3) 

RETR  (2) 

RTMSG 

RTVMSG 

OLD 

RECALL 

GET 

10.  Print  a  paper  copy  of  a  message.  PRINT  (4) 

PRT  (3) 
PRINTMSG 
PRTMSG 
PTMSG 

11.  Display  the  outgoing  message  queue.  QUEUE  (4) 

MSGQUE  (2) 

DSPYQUE 

DISPLAY 

OUTGOING 

OUTMSG 

12.  Transmit  a  message.  XMT  (4) 

XMIT  (3) 

TMT 

TRANSMIT 

TRANXMSG 

13.  PQS  syllabus  training,  programmed  texts.  PQS  (8) 

TRNG  (2) 
TRAING 
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14.  Thesaurus. 


THESAURUS  (5) 
THSRS 
TSURS 
THES 
THS 

15.  Cut  a  teletype  tape  of  a  message.  TTYTAPE  (3) 

TAPE  (2) 

TTY 

CUTTTYTAPE 

CUTTAPE 

REPERF 

16.  Conflgure  Comm  Central  equipment.  CONFIG  (4) 

CONFIGURE  (2) 

COMMCENT  STATUS 

REDSTAT 

COMM 

CCC 

17.  Spell  check  a  message.  SPELLCHECK  (5) 

SPELL  (4) 

SPELLCHK 

18.  Save  a  message.  SAVE  (6) 

SAVEMSG  (3) 

SMSG 

19.  Calendar  of  scheduled  events.  CALENDAR  (6) 

CAL  (2) 

CALDR 

20.  Backup  mission  data  from  the  hard  BACKUP  (3) 

disk  to  floppy  diskette.  FLOPPY  (2) 

BACKMSG 

BKP 

DISK 

COPY 
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APPENDIX  C.5 


PICTURE  ELEMENTS  FROM  ICON  PRODUCTION  METHOD 
Task  description  Picture  elements  (responses) 

1.  Preflight  tests  and  checklists. 

Checkmark  (5) 

Pocket  checklist  (2) 

Aircraft  with  checkmark 
Check  on  paper 

2.  Display  message  catalog  (flies). 

Paper  files  (5) 

Paper  with  arrow  (2) 

Book  (2) 

Rolodex  file 


3.  Create  a  message. 

Paper (2) 

Person  with  paper 
Person  with  question  mark 
Pen,  ink,  and  paper 
Pen  and  ink 
Open  book 

4.  Operational  and  maintenance  publications,  NATOPS. 

Books  (3) 

Paper  with  zzz’s 

5.  General  help  index  for  MMPS  software. 

Question  mark  (2) 

Help  cross  (2) 

Person  with  question  mark 
S-O-S  in  morse  code 

6.  Aircraft  systems  readiness  status  information. 

Aircraft  outline  (5) 

Aircraft  with  up/down  arrows  (2) 
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7. 


Block  and  cut  a  section  of  a  message. 

Scissors  with  message  (3) 

Scissors  (2) 

Message  with  highlights  (2) 

Block 

8.  TACAMO  aircrew  basing  information. 

Map  of  the  United  States  (3) 

Motel  (3) 

Aircraft  with  message 
Bed 

Telephone 

Stopsign 

9.  Retrieve  an  existing  message. 

Computer,  arrow  to  paper  (2) 

Dog  with  bone  (2) 

Computer,  arrow  to  monitor 
Monitor  with  "R" 

Hand  grabbing 

10.  Print  a  paper  copy  of  a  message. 

Printer  with  paper  output  (6) 

Stack  of  paper 
Piece  of  paper 
Arrow  to  paper 

11.  Display  the  outgoing  message  queue. 

Letter  "Q"  (2) 

Aircraft,  lightning  bolt,  question  mark 

Satellite  dish 

Message 

12.  Transmit  a  message. 

Lightning  bolts  (3) 

Satellite  dish  with  lightning  bolts  (2) 
Circuit  symbols  (2) 

Aircraft  with  lightning  bolt 
Drogue  with  arrows 
Drogue 
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13.  PQS  syllabus  training,  programmed  texts. 

"PQS"  in  box 
Clipboard 


14.  Thesaurus. 

Book  with  "T"  or  thesaurus 
Book  with  question  mark 

15.  Cut  a  teletype  tape  of  a  message. 

Teletype  tape  and  scissors  (2) 
Teletype  tape  (3) 
Reperforator  (2) 

Roll  of  teletype  tape 

16.  Configure  Comm  Central  equipment. 

Comm  central  likeness  (4) 
Patch  panel 
Oldtime  radio 

Face  with  "CONFIG"  bubble 
Aircraft  with  up/down  bubble 

17.  Spell  check  a  message. 

"SPELL"  with  checkmark  (3) 
Spelling  bee 
Book  with  checkmark 
CAT/KAT  misspelling 


18.  Save  a  message. 

Monitor,  arrow  to  computer  (2) 
Message,  anow  to  computer 
Diskette 
Life  ring 

Baseball  diamond 

19.  Calendar  of  scheduled  events. 

Calendar  month  (10) 

20.  Backup  mission  data  from  the  hard  disk  to  floppy  diskette. 

Diskette  (5) 

Computer,  arrow  to  diskette 
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APPENDIX  D 


ICON  CANDIDATES 
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APPENDIX  D.2 


TACAMO  ICON  SET  REVISIONS 


1.  Transmit  a  message.  Original  ideas  were  abundant  for  this  icon,  from  a  picture 
of  the  drogue  at  the  end  of  the  transmitting  wire  to  a  telegram  symbol.  The  arrow 
bolting  downward  from  the  silhouette  of  a  TACAMO  E-6A  aircraft  was  chosen  to 
signify  the  transmission  of  a  message  from  the  TACAMO  aircraft  The  arrow  was 
used  as  a  common  attribute  in  icons  used  to  depict  message  movement  in  this  icon 
set. 

2.  Queue  of  outgoing  messages.  A  capital  "Q"  was  the  clearest  representation 
available  to  represent  this  abstract  icon,  which  will  be  used  to  check  the  flow  of 
outgoing  messages.  Various  fonts  were  considered,  but  a  DrawPerfect’s  WP  Century 
Schoolbook  font  gave  the  "Q"  its  distinctive  tail. 

3.  Create  a  message.  The  scroll  and  quill  design  was  borrowed  from  the  Navy’s 
enlisted  journalist’s  rating  badge.  Pens,  inks,  and  even  word  processor’s  were 
considered,  but  the  rating  badge  drew  high  approval  marks  from  Navy  personnel. 

4.  Display  message  files.  The  old  TMPS  terminology  for  this  icon  would  be  "display 
message  catalog."  Drawings  and  ideas  were  rather  weak  on  this  task  description,  so 
an  office  metaphor  of  a  group  of  file  folders  was  chosen  over  a  picture  of  a  file 
cabinet. 

5.  Print  a  message.  This  icon  show  a  printer  with  paper  feeding  out  of  the  top  of 
it.  This  version  may  have  drawn  high  support  because  it  closely  resembles  the 
teletype  printer  currently  used  aboard  the  TACAMO  aircraft. 

6.  Punch  a  teletype  tape.  A  spool  of  teletype  tape  was  drawn  for  this  icon.  It  was 
reversed  on  the  y-axis  to  prevent  confusion  with  the  question  mark  used  for  the  help 
icon. 

7.  Save  and  retrieve  message.  These  icons  also  employ  arrows.  Save  signifying  a 
message  being  stored  in  a  computer’s  fixed  hard  drive.  Retrieve  signifying  a  message 
being  retrieved  from  computer  memory.  Arrow  directions  were  modified  between 
the  save,  retrieve,  and  backup  icons  to  reduce  confusion  from  similar  arrow  types. 

8.  Spellcheck  a  message.  This  task  description  was  described  by  most  as  the 
toughest  to  represent  pictorially.  The  checkmark  seemed  natural,  but  glasses  were 
added  to  represent  additional  focus. 
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9.  Cut  a  block  of  a  message.  The  scissors  cutting  into  a  block  depicts  a  common 
text  editing  function. 

10.  Status  of  aircraft/equipment.  The  thumbs  up  and  thumbs  down  with  question 
marks  for  uncertainty  was  chosen  for  this  icon.  The  thumbs  were  chosen  to  replace 
arrows  to  lessen  confusion  with  arrows  representing  message  movement 

11.  Configure  Comm  Central.  This  icon  shows  three  plugged-in  patch  panel 
connections,  similar  to  communication  patches  made  aboard  the  TACAMO  aircraft 

12.  Preflight  checklists.  The  aviator’s  three-ring  checklist  with  a  large  check  through 
it  was  a  natural  for  this  icon. 

13.  Backup  message  to  diskette.  The  arrow  shows  a  message  being  stored  to  a  3.5" 
diskette,  the  size  of  diskette  anticipated  for  use  with  the  improved  MMPS. 

14.  Abbreviations  and  acronyms.  A  book  with  "ABC"  and  an  abbreviation  for  the 
Dictionary  of  Naval  Abbreviations  was  considered  for  this  icon  which  was  later 
dropped  from  the  study. 

15.  Publications  library.  A  set  of  books  is  shown  for  this  pictorially-based  icon 
drawn  from  Modley’s  (1976)  book  of  pictorial  symbols. 

16.  Maintenance  manuals.  A  separate  icon  for  use  by  the  flight  engineers  is 
designed  to  give  direct  access  to  the  aircraft’s  maintenance  and  equipment  manuals. 
The  wrench  inside  a  large  book  neatly  combines  both  ideas. 

17.  Training  and  qualifications.  An  open  book  signifying  studying  is  laid  across  a 
set  of  aircrew  wings  for  this  icon. 

18.  Help  and  Information.  The  question  mark,  a  universal  symbol  for  help  and 
information,  was  a  clear  selection  for  this  icon. 
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APPENDIX  D3 


DISCRIMINABILITY 

INDICES 

Icon 

XI 

X2 

25-Z 

Z 

DI 

HELP 

18 

0 

23 

2 

2.25 

TRANSMIT 

19 

16 

20 

5 

2.30 

SAVE 

19 

16 

20 

5 

2.30 

RETRIEVE 

19 

16 

20 

5 

2.30 

BACKUP 

19 

16 

20 

5 

2.30 

PREFLIGHT 

18 

17 

20 

5 

2.30 

QUEUE 

19 

0 

24 

1 

2.70 

FILES 

19 

0 

24 

1 

2.70 

TAPE 

19 

0 

24 

1 

2.70 

CONFIGURE 

19 

0 

24 

1 

2.70 

CALENDAR 

19 

0 

24 

1 

2.70 

BASES 

19 

0 

24 

1 

2.70 

CREATE 

19 

17 

21 

4 

2.75 

PRINT 

19 

17 

21 

4 

2.75 

PUBS 

19 

17 

21 

4 

2.75 

MAINT 

19 

17 

21 

4 

2.75 

TRAINING 

19 

17 

21 

4 

2.75 

CUT  BLOCK 

19 

18 

22 

3 

3.20 

SPELLCHECK 

19 

18 

22 

3 

3.20 

STATUS 

19 

18 

22 

3 

3.20 

c  =  20 

where  DI 

=  0.07[X1+X2+(25-Z) ] 

- 

0.31Z. 

REVISED  DISCRIMINABILITY  INDICES 

Icon 

XI 

X2 

23-Z 

Z 

DI 

HELP 

16 

0 

21 

2 

1 .83 

TRANSMIT 

17 

14 

18 

5 

1 .88 

SAVE 

17 

14 

18 

5 

1  .88 

RETRIEVE 

17 

14 

18 

5 

1  .88 

BACKUP 

17 

14 

18 

5 

1 .88 

PREFLIGHT 

16 

15 

18 

5 

1 .88 

QUEUE 

17 

0 

22 

1 

2.28 

FILES 

17 

0 

22 

1 

2.28 

TAPE 

17 

0 

22 

1 

2.28 

CONFIGURE 

17 

0 

22 

1 

2.28 

CREATE 

17 

15 

19 

4 

2.33 

PRINT 

17 

15  . 

19 

4 

2.33 

PUBS 

17 

15 

19 

4 

2.33 

MAINT 

17 

15 

19 

4 

2.33 

TRAINING 

17 

15 

19 

4 

2.33 

CUT  BLOCK 

17 

16 

20 

3 

2.78 

SPELLCHECK 

17 

16 

20 

3 

2.78 

STATUS 

17 

16 

20 

3 

2.78 

c  =  18  where  DI  =  0 . 07{ XI +X2+ (23 -2) ]  -  0.31Z. 
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APPENDIX  D.4 


CONFUSIBILITY  MATRIX 


Actual 
Stimulus  1 

2 

3 

4 

5 

6 

7 

Response 

(subject  selection) 

8  9  10  11  12  13 

14 

15 

16 

17 

18 

19 

20 

c 

>ther 

1 

100 

, 

! 

- 

r® 

0 

2 

95 

i — 

5 

3 

l - 

!  100 

1 _ 

1 - 

~ 

1 - 

0 

4 

n 

86 

r 

j 

14 

5 

5 

n 

76 

r 

10 

5 

5 

6 

5 

! 

95 

0 

7 

5 

95 

0 

8 

100 

0 

9 

100 

0 

10 

100 

0 

11 

5 

10 

5 

81 

0 

12 

100 

0 

13 

86 

14 

0 

14 

i 

1 

90 

; 

10 

15 

: 

' 

1 

| 

i 

1 

100 

0 

16 

100 

0 

17 

100 

0 

18 

14 

86 

0 

19 

1 

100 

0 

20 

100 

_ 

0 

This  matrix  represents  the  actual  stimuli  as  rows  and  the  responses  of  the 
subjects  as  columns.  Each  number  represents  a  percentage  of  the  subjects’ 
responses.  Entries  along  the  main  diagonal  represent  recognition  and  off-diagonal 
entries  represent  confusion. 
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APPENDIX  E 


ICON  DESIGN  FORMAT  EXPERIMENT 


I 
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APPENDIX  E.1 


TACAMO  AND  DISTRACTOR  ICONS  FOR  FINAL  TESTING 
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APPENDIX  E.2 


TIMESTAMP  PROGRAM 
STAMP. C 


#include  <dos.h> 

#include  <stdio.h> 

void  main(int  argc,  char  **argv) 

{ 

struct  timet.s; 

FILE  *fp; 

if((fp  =  fopen ("datafile",  "at"))==  NULL){ 
printf("FILE  did  not  open  correctly.  Check  memory  management"); 
exit(1); 

} 

if  (argc  !=  2){ 

printf ("Wrong  number  of  parameters  were  passed"); 
exit(1); 

} 

gettime(&t); 

fprintf (fp,  "%02d : %02d. %02d\t%s\n", 
t.ti_min,  t.ti_sec,  t.ti_hund, 
argv[1]); 

fclose(fp); 

} 


Disclaimer  The  reader  is  cautioned  that  this  program  may  not  have  been  exercised 
for  all  cases  of  interest.  While  every  effort  has  been  made,  within  the  time  available, 
to  ensure  that  it  is  free  of  computational  and  logic  errors,  it  cannot  be  considered 
validated.  Any  application  of  this  program  without  additional  verification  is  at  the 
risk  of  the  user. 
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APPENDIX  E3 


INSTRUCTIONS  FOR  ICON  DESIGN  FORMAT  EXPERIMENT 


Background.  The  Naval  Air  Development  Center  is  supporting  thesis  research  from 
the  Naval  Postgraduate  School  for  the  development  of  future  TACAMO 
communications  upgrades.  This  experiment  is  the  final  testing  stage  for  an  icon  set 
developed  for  use  aboard  the  TACAMO  airborne  communications  platform.  The 
experiment  consists  of  five  phases  conducted  on  a  personal  computer: 

1.  Initial  Icon  Training 

2.  Review  Training 

3.  Baseline  Testing 

4.  Additional  Review  Training 

5.  Icon  Recognition  Testing 

Your  participation  will  indicate  how  well  each  of  the  developed  icons  is  associated 
with  its  intended  purpose. 

Purpose.  This  experiment  will  be  used  to  measure  the  effectiveness  of  the  icon  set 
to  increase  performance  while  reducing  errors. 

Icon  Training.  The  initial  icon  training  phase  wil  present  eighteen  icons  with  task 
descriptions.  All  training  is  self-paced  and  typically  advances  with  the  pressing  of  the 
spacebar.  Review  training  presents  the  icons  again  with  abbreviated  task 
descriptions. 

Baseline  Testing.  Baseline  testing  uses  the  numeric  keypad  keys  one  through  nine 
with  a  simple  search  for  a  black  block  in  the  3x3  matrix.  The  index  finger/forefinger 
of  the  right  hand  will  start  at  a  specified  point  beneath  the  numeric  keypad  and 
should  return  to  that  point  before  continuing. 

Primary  Task.  In  the  main  icon  recognition  test,  a  text-only  screen  asks  you  to  select 
the  icon  which  best  represents  an  abbreviated  task  description.  When  you  are  ready 
to  continue,  pressing  the  spacebar  will  present  a  3x3  icon  matrix.  Press  the  key  of 
the  numeric  keypad  which  corresponds  to  the  described  icon  and  return  your  finger 
to  the  starting  point.  In  addition  to  the  eighteen  TACAMO  icons,  eighteen  arbitrary 
icons  have  been  randomly  included  in  the  search  matrices.  You  are  asked  to  work 
as  accurately  and  quickly  as  possible.  If  you  have  any  questions,  please  ask  them  at 
this  time. 


APPENDIX  E.4 


Age: 


Sex:  M  F 


Primary  hand:  L  R 


Education: 

High  School/GED 
Some  College  Courses 

Associate’s  Degree _ 

College  Degree _ 

Postgraduate _ 


Computer  experience  (circle  as  many  as  applicable): 

IBM  PCs  Macintosh  Mainframe  terminals 

Amiga  Commodore  Other _ 
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APPENDIX  E.5 


ICON  TASK  DESCRIPTIONS 

1.  Transmit  a  message. 

2.  Queue  of  outgoing  messages. 

3.  Display  message  files. 

4.  Create  a  message. 

5.  Save  message  to  computer  storage. 

6.  Retrieve  message  to  display  monitor. 

7.  Print  a  message. 

8.  Punch  a  teletype  tape. 

9.  Cut  a  block  of  a  message. 

10.  Spellcheck  a  message 

11.  Back  up  a  message  to  diskette. 

12.  Preflight  checklists. 

13.  Status  of  aircraft/equipment. 

14.  Configure  patch  panel. 

15.  Publications  library. 

16.  Maintenance  manual. 

17.  Training  and  qualifications. 

18.  Help. 
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APPENDIX  E.6 


EFFECTS  OF  BASELINE  SUBTRACTION 

E.6.A.  BASELINE  REACTION  TIME  DATA  BEFORE  SUBTRACTION 

OF  AVERAGE  REACTION  TIME 


-  + . + . + . + . + . + . Sec 

4.00  4.20  4.40  4.60  4.80  5.00 

Each  dot  represents  7  data  points. 


E.6.B.  BASELINE  REACTION  TIME  DATA  AFTER  SUBTRACTION 

OF  AVERAGE  REACTION  TIME 


-  -  -  -  + 
-0.15 


—  + 
0.00 


■  •••«•••••••  ••  •  •  • 

- + . + . + . Sec 

0.15  0.30  0.45  0.60 

Each  dot  represents  3  data  points. 
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E.6.C. 


DOTPLOTS  OF  SUBJECTS’  BASELINE  REACTION  TIME 
BEFORE  SUBTRACTING  AVERAGE  REACTION  TIME 


*♦ . + . ♦ . . ♦ . ♦ . + . Sec 

4.00  4.20  4.40  4.60  4.80  5.00 
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-+ . Sec 


. Sec 


-♦ . Sec 


-+ . Sec 


-+ . Sec 


-+ . Sec 


•+ . Sec 


•+ . Sec 


-+ . Sec 


•+ . Sec 


•  + . Sec 


-+ . + . + . ♦ . ♦ . + . Sec 

4.00  4.20  4.40  4.60  4.80  5.00 
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E.6.D.  DOTPLOTS  OF  SUBJECTS’  BASELINE  REACTION  TIME 
AFTER  SUBTRACTING  AVERAGE  REACTIOM  TIME 


— + . +- 

-0.15  0.00 


- + . ♦- 

0.15  0.30 


- ♦ . Sec 

0.45  0.60 
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SUBJ 

19 


SUBJ 

20 


SUBJ 

21 


SUBJ 

22 


SUBJ 

23 

SUBJ 

24 

SUBJ 

25 


SUBJ 

26 


SUBJ 

27 

SUBJ 

28 


SUBJ 

29 


SUBJ 

30 


SUBJ 

31 


SUBJ 

32 


SUBJ 

33 


SUBJ 

34 


SUBJ 

35 


SUBJ 

36 


-Sec 

•Sec 

-Sec 

-Sec 

-Sec 

•Sec 

-Sec 

-Sec 

-Sec 

-Sec 

-Sec 

-Sec 

-Sec 

-Sec 

-Sec 


-Sec 


;  ,  ;  i ; 

-0.15 

0.00 

0.15 

0.30 

0.45 

0.60 
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APPENDIX  E.7 


95%  SIGN  CONFIDENCE  INTERVALS  FOR  IDENTIFICATION  TIME 

E.7  A.  LABELED  AND  UNLABELED  TACAMO  ICONS  COMBINED 

Icon  95%  Confidence  Interval 


1 

-1+  I---** 

2 

-1+  I--  *  0 

3 

---I  ♦  I 

5 

---I  ♦  I . 

6 

--I  +  I---  *  0 

7 

---1+  I--*  *•  0 

8 

---1+  I . 

9 

•I  +  I-. 

10 

•  -l+I —  0 

11 

--I  ♦  I . 

12 

•  I*  I —  • 

13 

•  •I  ♦  I . 

14 

-I  ♦! - 

15 

. I  *  * 

16 

---I  +  I .  •  • 

17 

--I  +  I . 

18 

-I+I--  * 

•f . 

0.0  2.5  5.0  7.5  10.0  12.5 
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E.7.B. 


UNLABELED  TACAMO  ICONS 


Icon 

95%  Confidence 

Interval 

IP 

I+I-  * 

2P 

-I+I--- 

3P 

---1+ 

I- 

4P 

-•••I  4-  I  *  -  ■ 

5P 

--i  +  i-- 

6P 

-I  +1- 

7P 

I--- 

• 

8P 

••I  +  1 . 

9P 

-I  4  I-- 

10P 

--  +  I-  0 

IIP 

-I  +  I-- 

12P 

---I+1--  • 

13P 

-•-I  ♦  I . 

* 

14P 

-•I*  I--- 

15P 

---I  ♦  I-- 

• 

16P 

--I  +1 

• 

• 

17P 

--I+  I-- 

18P 

-I+I- 

0.0  2.5 

5.0 

99 


♦  m 


E.7.C, 


LABELED  TACAMO  ICONS 


Icon  95%  Confidence  Interval 

1L  -1+  I - 

2L  -I  +I--  *  • 

3L  ---I  +  I .  *  * 

4L  - 1  +  I - 

5L  ---I  4  I . 

6L  --I  +  I-- 

7L  - 1+  1 .  0 

8L  ---1+  I . * 

9L  -I  4  I-- 

10L  -I+I--- 

1 1  L  - 1+  I- 

12L  ---I  4  I . 

13L  --I  4  I . 

14L  -I  +1- 

15L  - 1  +1- 

161  ---I  +  I .  * 

17L  --I  +  I . • 

18L  -I+I-  * 

4 . 4 . 4 . 4 . 4 . 4 . C9 

0.0  2.5  5.0  7.5  10.0  12.5 
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